Photo by Elliot Teo on Unsplash

Introduction to Multimodality and Multimedia

Every day, we see arguments made with images, sound, and text. These forms of communication are called modes, and any argument that draws on them is by definition multimodal.

Common examples of multimodal arguments include posters, fliers, billboards, graphs, cartoons, instruction manuals, magazines, websites, and videos. They use many of the same strategies as written arguments, including the appeals (ethos, logos, pathos). A multimodal argument simply represents ideas and evidence through different means.

Multimodal composition enables you to engage audiences in different ways in the same argument. You might already know that people process information in different ways. Some members of your audience prefer to engage with ideas through images, and others think and communicate better aurally rather textually. When you blend modes, you’re creating dynamic content that maximizes the chance to convey your ideas.

Infographics

Infographics serve as an especially effective form of multimodal rhetoric. They often accompany articles and op-ed columns, highlighting important facts and statistics through visual representation. This form of communication has become increasingly popular in recent years.

Research compiled in an infographic by NeoMam Studios argues that the popularity of visual representations owes to own tendency to process information better in visual forms. Incorporating visuals into a verbal argument increases the level of comprehension, engagement, and receptiveness. They’re more accessible, and therefore more likely to be shared on social media. Consider this example from The Atlantic, an infographic that explains why smartphones cost so much:

As this example shows, an infographic arranges information in a compelling way to tell a story. Consider the choice of font and background, as well as the use of color to emphasize information the author wants you to pay attention to. This information would take a long time to explain textually. But with a visual, it’s easy to see that each component of a smartphone costs relatively little. Meanwhile, smartphone companies like Apple make a substantial profit–much more than the average person might realize.

You can use infographics to do more than simply show facts and statistics. Many writers have designed infographics in order to represent complex processes and systems. They also work well to show chronologies, or networks and relationships between individuals and events.

You can study this infographic on the infastructure of the Internet by Telx.com, a data center company. Consider how the designer has arranged text and visuals to explain how data circulates through the web.

Videos

Videos offer another powerful form of multimodal rhetoric. As blends of all three modes, their content and purpose can range widely. Many activists record YouTube videos where they speak directly into the camera, addressing readers directly. They might also record and compose messages that help spread awareness of social causes, and build community among people located across cities and regions.

For example, the “Liberal Redneck” Trae Crowder has posted dozens of videos from his own back yard, where he comments on socio-political and cultural issues. Specifically, Crowder’s rhetoric aims to show that supporting progressive social causes doesn’t necessarily conflict with Southern values. Another vlogger, Kat Blaque, offers commentary and analysis on a range of political and cultural controversies.

Others collect and post footage of the actions and behaviors they seek to protest–including examples of discrimination and harassment. One particular nonprofit organization, Hollaback, has sponsored a number of amateur videos in which women illustrate the frustrations of street harrassment and catcalling through video documentation.

Video can sometimes exceed a single photograph or text’s ability to create a sense of context, immediacy, and urgency. Even smartphone footage can give viewers a vivid or visceral experience, as if they were present. In the case of Hollaback’s videos, the footage immerses viewers in the authors’ environment in ways that words or images themselves might not otherwise give full justice.

Here, you see all three modes working together to construct meaning for audiences. First, video and audio provides clear evidence to support Hollaback’s arguments about the pervasiveness of street harassment. The captioning helps to clarify the audio, as well as making the dialogue accessible to those with disabilities. Finally, consider that this video is embedded in YouTube, a site which enables textual interaction and hyperlinking.

Video editing tools have become remarkably sophisticated in recent years, allowing anyone to record, trim, and arrange clips from several sources for a range of purposes. Careful editing can give videographers and vloggers a great deal of control over how to best present their material. As the cat-calling videos illustrate, it’s possible to sample several hours worth of video and construct them into a short narrative of a few minutes. You can also add voice overs to the clips in order to add commentary and continuity. Many of these tools are intuitive, and some of them work on your phone.

With the right apps, you could even compose short documentaries on important topics of nearly professional quality.

Mode, Medium, and Affordance

Infographics and videos are both examples of medium, the method of combining modes in order to convey ideas. Not every medium combines all three modes. Some of them, like print journalism, mainly use textual and visual. By contrast, video serves as a medium that combines all three modes–visual (images), aural (sound), and textual (words). A song would also be considered a medium, and it primarily combines words and sound. Meanwhile, a music video blends all three modes. As another medium, blog posts often combine words and images, but don’t always use sound. Of course, a blogger could choose to record their post as a podcast, and upload it to their post.

In short, a mode describes a way of discourse. A medium describes the product containing a mixture of modes.

Every medium offers different affordances and constraints. An affordance refers to an opportunity, a unique aspect of the medium not necessarily found in other media. A constraint refers to a limitation of a specific medium.  For example, music can convey emotions through the combination of lyrics and instrumental performance in ways that a text simply can’t by itself. On the other hand, an article may work better at explaining a complicated idea to different audiences. A report often doesn’t illicit strong emotions, but they .

Because of these affordances, some media are ideally suited for specific rhetorical situations. Instruction manuals with diagrams do the best job satisfying a furniture company’s need to explain to consumers how to assemble their purchase. A song wouldn’t work as well. However, a video showing someone constructing the furniture might help to supplement written instructions.

Modality & Fit

Infographics and videos are just two types of multimodal rhetoric. Other examples include Prezi slide shows, white papers, photo-essays, podcasts, and even PowerPoint presentations. In each case, someone is drawing on different modes to express their ideas.

Every act of communication has a multimodal component. You can even think of a plain text document as multimodal. Although text is the primary medium, writers and page designers often deliberate about the layout of an article–including font type and size, paragraphing, and white space. They also make decisions about how to present featured sections of text and quotations through the use of indentation, bolding, and italics.

When you compose your own multimodal arguments, think about your purpose and what modes and mediums will best help you. What audiences do you want to reach, and what information do you want to convey? What affordances and constraints of different modes and media can help you best achieve your goals? You might even decide to experiment with a number of media before settling on the one that fits best.