When image and text meet: teaching with visual and multimodal texts

Jon Callow

Arial view of two isolated forest areas, forming a pair of lungs in shape, with one partly deforested within

Original caption: ‘Before it's too late.’ Image source: World Wildlife Federation (WWF) Climate Change

A seamless world

Being immersed in a visual culture, we can sometimes not even notice how we interact with images and words. The photo above of the logged forest catches our eye, summoning the image of a pair of lungs. The poster comes together with the conservation group’s logo and the tag line ‘Before it's too late’, making a powerful point.

Engaging with various texts can sometimes seem a simple, everyday act, belying the complexity involved. Until something catches our eye, jarring us, or delighting us with its simplicity, its beauty or its drama, we may not stop and reflect more deeply about what we have seen or experienced. It is when we do stop that we wonder how the author, artist or designer communicated their message so effectively.

This PETAA Paper will:

  • explore multimodality, with a focus on image and text
  • review key visual metalanguage
  • demonstrate how to plan effective modelled viewing lessons using picture books
  • present examples of annotated picture book images to guide classroom practice.  

Related publication

A small cairn of river stones witth book title

Details: Book | eBook

See additional support for educators using this title.

The concept of multimodal texts is integrated in the Australian Curriculum: English, across the language, literature and literacy strands. From understanding social contexts, expressing and developing ideas through language and literature, to creating new texts, the multimodal nature of English is deeply embedded in the rationale and strands of the curriculum. Teamed with a knowledge of written language is the inclusion of a visual language thread (ACARA, 2011:66), as well as assessment linking to multimodal texts, expressed in the achievement standards from Foundation to Year 10.

A key concept to understand when working with multimodal texts is that the whole is greater than the sum of the parts. This means that we need to make sure we don’t just read a text like a picture book, talk about one or two literary features, discuss which picture we like the best then move on to the next activity. Multimodal texts make meaning because all the elements work together to create a whole text. This is the case for picture books, using only image and text, or for video and multimedia, where image, gesture, movement, words and sound all work together to create the final piece.

Consider the image for the advertisement below. Initially we are drawn to the fin on the left, perhaps reacting with a sense of fear. The caption reinforces this, yet when we look to the right, the fin is gone, but the caption suggests this is ‘more horrifying’. Isn’t this a contradiction? Is it because the shark is under the water? What could the advertisement mean? Reading the text on the bottom, the purpose becomes clearer — the loss of wildlife has an impact on the ecosystem. This is the same ecosystem on which humans also depend. The poster cleverly plays with fear of sharks and the juxtaposition of image and word, in order to convey its message.

 Imagine with shark fin in water in left panel and fin absent in water on right

Caption: ‘Exploiting the ecosystem also threatens human lives.’  Image source: World Wildlife Federation (WWF) Sharks

We can learn through multimodal texts, where our emotions and intellect are engaged but we can also learn about how texts like this work, by understanding how both the written and visual elements work (Callow, 1999; Kress & Van Leeuwen, 2006).

Visually the fin is the most salient element in the poster, with the adjective ‘horrifying’ shaping our interpretation and response, as well as augmenting the image. Reading left to right we move from the shark being present — the ‘given’, to its apparently sudden disappearance — the ‘new’. This new information is possibly more unsettling. The dark blue colours further add to a fearful atmosphere, as well as the fact that the fin in the first photo (and by association the shark) is framed to appear quite close to us as the viewer. The placement of the large photos at the top, with the more detailed text and logo down the bottom is common. Many advertisements put their main ideas at the top, considered the ‘ideal’ in western culture, while the more everyday aspects of the product, such as the details, or specific features, are often at the bottom, symbolically grounded in what we can term the ‘real’. The written text here is also powerful — ‘Exploiting the ecosystem also threatens human lives’. Strong action verbs like exploiting, and threatens create an immediate emotional reaction, especially when linked to the noun group human lives. From a critical literacy aspect, we can also ask about the world-view of this organisation as advocates for wildlife. Is the information true — would the loss of sharks really impact us as humans? Should we think about supporting this organisation? Reflecting critically allows us the space to step back and consider the points of view presented. (Callow, 2008.) Exploring multimodal texts can range from advertisements, posters and videos to one of the mainstays of any English classroom — the picture book.

Modelled viewing and picture books

Teaching viewing using picture books provides both exposure to quality literature as well as a stimulating variety of visual images and techniques. The following provides a guide for choosing and using pictures books with a focus on the visual and written text interplay.

Read and enjoy: Having chosen a piece of quality literature, make sure you read it and enjoy it, as your own knowledge and enthusiasm is important and will always positively influence your teaching.

Choose a key page from the story: There are often key moments in a story, which are represented in word and image. When planning, choose a key page, note down the main ideas and the visual and verbal features that are important in creating the ideas and feel of the page (refer to the samples below).

Literary themes and features: As you are teaching a whole text, consideration of the broader literary features should inform your lesson planning (Saxby,1997). These elements are also key aspects of the Australian Curriculum: English. Features include:

  • Theme — ideas that hold the story together, such as friendship, survival, family, courage
  • Plot — traditional narrative structures or variations on this
  • Characters — major and minor characters, heroes and villains, animals as characters
  • Point of view — first person or third person
  • Place or setting — realistic (urban, city, bush), fantasy or imaginary, historical.

Table 1 below gives a summary of features that may be present. Only choose the features that are relevant or strongly represented on the page.

(Concepts of print and screen; Visual Language*)
Multimodal meanings
Guiding principles to consider
when image and text meet
(Structures of different types of texts; Sentence and clause level grammar; Word level grammar; Vocabulary*)

Actions and settings – how line and shape represent people, objects and events, including speech and thought
Conceptual and symbolic – images showing ideas and conceptual themes

Right facing arrow

In the context of the unfolding story, how does the page develop the characters, story events and build themes and concepts?

How do the visual and written work together?
Events and Ideas

Sentence and clause structures
– building happenings and ideas through different verb types and adverbs/ adverbials – description of characters and setting through noun groups and adjectives

Left facing arrow
Reacting and relating

Shot distance – long, mid or close
– high, low or eye level
Gaze of characters
– offers and demands
– moods and symbolism
Shapes and textures
and their impact
Realism of images – photo, artwork, drawings, diagrams  

How do the visual and written text shape our feelings and reactions to the characters?

Do we react differently to the visuals than to the written text?

Do the literary features complement or contradict the visual elements?

Statements, questions and commands
Attitudes and opinions
Modality – degrees of certainty
Literary features – alliteration , repetition, rhyme, personification, metaphor; interesting vocabulary choices

Design and layout

Balance – symmetry and contrast of elements
Top/bottom, left/right, centre/margin – placement of elements
Salience – what attracts attention
Reading paths – vectors directing our gaze
Framing – how text and image may frame each other; image cropping 

How does the layout of the visual and written text influence the reader/viewer?

Does the layout privilege particular visual or written elements?

Why might the author/illustrator have made these choices?
Text organisation

Genre/ text type
– literary (narrative, ballad, recount etc.) or factual
Sentence order and structure

Table 1: Multimodal meanings — image and text

* denotes thread from Australian Curriculum: English. For more details on visual grammar, see Callow, 1999, and for more details on written grammar, see Derewianka, 2011

Scaffolding multimodal meanings

The middle column in the table brings together the visual and written aspects of the texts to ensure that the multimodal meanings are explored. Rather then just listing specific visual features and grammatical aspects, we need to take that extra step and explore how they work together (Walsh, 2010). Pedagogically, this will include reading the whole text, revisiting key pages and events in the story, explicitly teaching an aspect of the visual and written text and how they work together, then consolidating the concepts and skills in a concluding activity.

In a modelled viewing lesson (whole class or small group), a teacher demonstrates key visual and written features as well as involves their learners in a variety of discussions and hands-on activities. Within one lesson, a teacher may move across a scaffolding continuum, from more teacher-regulated practices (reading the text, making statements about key features, asking focused questions about features) to more scaffolded activities, where students take some responsibility for practising the demonstrated skills or knowledge (Hammond, 2001). Ideas for activities can include:

  • sorting a selection of images into shot distances or types of angles
  • listing key features such as salient points, characters that demand viewer attention or effective use of colour
  • re-drawing a character so that they either look at the viewer (demand)or away from them (offer)
  • creating a frozen moment scene from the story and using a digital camera to experiment with angles, framing or shot distance.

Example 1: Fox

Consider the example from the book Fox, which shows an overview of a chosen double page spread. The following ideas are suggestions for planning a modelled viewing lesson.

Fox and magpie proximate on coverFox by Margaret Wild and Ron Brooks

Themes: Friendship, loyalty, betrayal

Synopsis: A fable of friendship and treachery where Dog and Magpie’s relationship is challenged by Fox’s arrival in the bush.

Key features and context of image: As the orientation moves into the complication of this narrative, Fox arrives after Dog and Magpie have become close friends. A feral animal, he doesn’t belong either in the bush or in Dog and Magpie’s friendship. This page foreshadows the betrayal by Magpie, with Fox coming between the two friends, visually and in the written text. Fox’s flattery sees Magpie initially recoil in fear but her feelings change as his words influence her in the unfolding story.

Before reading

Building prior knowledge: Discuss the cover, finding out what preconceptions learners have about foxes. List words associated with foxes.
Explain the concept of a fable: a short story with a moral

Prediction: Could this Fox be a hero or villain? Look at the endpapers — what does the harsh red paint suggest? Why is there a dog with a magpie in its mouth before the title page?

During reading: Have a balanced number of questions and think-aloud statements that allow the story to unfold, as well as prompt comprehension for events and themes.

Think aloud statements and questions

Dog and Magpie build a strong friendship at the start, as Dog is faithful and kind.

  • Why is fox always watching?
  • How does he tempt Magpie?
  • Should Magpie go with Fox?

The descriptions of Fox as having ‘haunted eyes’ and a ‘rich red coat’ are very effective.

  • Why does Fox leave her in the desert?
  • Is Fox’s final cry a scream of triumph or despair?

After viewing the final endpapers, discuss whether Magpie gets home safely or not.


Multimodal meanings

Magpie is pulling away, laying on back, showing her fear. Dog is looking off the page, appearing trusting or possibly naive? Fox stares at Magpie — are his intentions friendly or evil?

Fox says one thing but does he mean another? Does his image match his words or contradict them? What do the words and image of Dog and Magpie suggest about their future friendship?
Events and Ideas

“You looked extraordinary” — a flattering statement but is Fox telling the truth? Dog beams but Magpie shrinks away — use of ‘but’ joining the clauses shows contrast between the two characters.

Reacting and relating

The colours give an earthy feel, with Fox’s red standing out as he dominates the page. Red is symbolic of danger, influencing our emotional response, as does the messy, handwritten lettering.

How does the use of colour build up our feelings about each character? Are the strong verb choices reflected in the image? Or do the images tell us something more?

Strong verb choices build tension in the story. Magpie shrinks away — shows her fear Dog beams — shows his innocence Fox staring at her burnt wing — suggests evil

Design and layout

Fox and Dog on the left, opposite Magpie on the right, gives an unbalanced feel. Magpie is now separated, small and vulnerable at the bottom of the page. Fox is at the top, in the ideal position, appearing dominant. Fox’s speech box is vertical. Both Fox and his speech box come between Dog and Magpie. His words are the centre of the page and the centre of attention too.

Why did Ron Brooks put the speech box vertically? Because it is askew, does it make the words in it mean something different? (look for other examples in the story where the text isn’t horizontal. What does it signal about the characters’ relationships?)
Text organisation

No clear paragraphing or text layout adds to confusion and the conflict that is growing. Fox’s words and actions create fear in Magpie.

Table 2: Multimodal meanings — an analysis of Fox

Circle each character's eyes — who is looking at who? Do they look at us?

Fox and his words are in the centre and at the top of the page. Both are framing Magpie, trapping her in the corner. How is Magpie reacting? How do we feel about Magpie?

Double page spread from Fox (text below)

Text on pages: But dog says, “Welcome. We can  offer you food and shelter.” “Thank you,” says Fox. “I saw you running this morning. You looked extraordinary.” Dog beams, but Magpie shrinks away. She feels Fox staring at her burnt wing.

Draw a line down the middle of the page. Dog and Fox are on the left, separated from Magpie. Why might Ron Brooks have done this?

Strong vector from Fox’s gaze — who has the power? Box the powerful verbs such as beams, shrinks, staring. Do they complement or contrast what we see?

Teaching strategies

Show the double page spread, either using the book or projected on a screen or an interactive whiteboard.

Vectors and gaze: Ask a student to trace along the gaze of Fox to Magpie (or draw an arrow on the interactive whiteboard). Re-read Fox’s statement. Pose some of the questions in the multimodal meanings column (see Table 2).

Colour: Discuss possible symbolic meanings of the colour red — danger, power, strength, romance, passion. Do any of these apply to Fox?

Layout: Discuss Fox’s flattery. Point out how his words and body are curving around Magpie. How does she react? How do we feel about Magpie?

Reactions: Circle or label the eyes. Do they look at us (demand) or elsewhere (offer)? What does this tell us about the character’s feelings or intent?

Written grammar: Box some of the strong verbs and discuss their impact using the multimodal meanings questions.

Key narrative point: How do the text and pictures mark a change in the story? How does this scene change the friendship between Magpie and Dog? How does it make us feel about each character?

Visual literacy an interactive whiteboards

The interactive tools and techniques available on interactive white boards extend the creative ways of teaching about multimodal and visual texts (see Goodwin, K. e-update 017, 2011)

Pens and highlighters: Ask students to annotate an image, circle key features or draw vectors and reading paths.

Drag and drop: Match labels to parts of the page such as description of angles or shot distances, noun groups or verb types.

Manipulate elements: Using elements from a page (such as characters or objects), re-design the page to tell a different story, re-size them to make one element salient, or create text boxes to manipulate the font, size and the position of text on the screen. 

In the classroom — Example 2, Clancy & Millie and the Very Fine House

Clancy & Millie and the Very Fine House by Libby Gleeson and Freya Blackwood

Two kids building a house of cardboard bxoes as high as house roofs and chimneys

Themes: friendship, play, imagination

Synopsis: Clancy moves to a new house, which is much too big and very lonely. However, Clancy meets Millie and together they find friendship, building towers, trains and houses for the three little pigs.

Key features and context of image: The second double page spread of the book below contrasts strongly with the first more colourful one, where the three little pigs cavort across the sky. As orientation blends into complication, this page plays with angles, proportions and colour, placing Clancy in a home that is overwhelming in its size and loneliness. Visually this pattern is repeated over the next four pages.

Can you see a wolf in the clouds? What feel does he give?

Circle Clancy and his parents — how big or small are they compared to the house? Why might they be drawn this way?

Double page spread from Clancyy and Millie ... text below

Text on left of double-page spread: ... to this house. ‘I love my new home,’ says Clancy’s mother. ‘It’s the best house.’ ‘It’s a very fine dwelling,’ says Clancy’s father. ‘It’s too big,’ whispers Clancy.

Box the noun groups describing the house — compare them to the pictures. Why might they be different? List adjectives to describe the houses in the picture.

Trace or draw along the vertical lines of the houses — how do you think the angle of the houses makes the family feel? A low angle shot can often make the viewer feel powerless.

Teaching strategies

After having read the book, come back to the first two pages. Use the annotated image to explore angle, size and descriptions.

Vectors and gaze: Circle Clancy and his parents. Trace the gaze of their eyes. What are they looking at? Why?

Reactions: Clancy’s parents sound confident when they describe the house. Do they look confident?

Key narrative point: How do the story and pictures tell us about Clancy’s feelings? Are the next pages similar? Keep reading to see if there is a pattern.

Table 3 below presents the multimodal meanings that can be read into this text. 


Multimodal meanings

Clancy and his parents, appearing very small, stand outside and gaze upwards at the imposing new house. The cloudy shape of a pig leaps back to the previous page, with the faint outline of a wolf chasing it.

The written text provides the dialogue here, while the visual gives more information about the characters. What do the pictures tell us about what they are doing and where they are looking?
Events and Ideas

The second part of the sentence from the previous page (Clancy has moved from this house) is completed here, to reinforce the contrast between homes (… to this house).


Reacting and relating

Like the characters, we are at a low angle, with the buildings purposely drawn with tall, angled walls leaning ominously forward. Using a long shot, Clancy and his parents appear small and powerless. The colours are drab with dirty browns, further affecting our feelings towards the house.

How do the colours make us react? Why do you think Freya Blackwood drew the buildings on an angle? We are low down, looking up at them — how does it make us feel? How does the contrast between the parents’ words and the picture make us feel?

Each character makes a statement about the house. Clancy’s parents describe the house as new, best, and a very fine dwelling. Tellingly, Clancy whispers rather than says that it is ‘too big’. Clancy’s mother’s positive attitude and comments continue over the next four pages, while Clancy yearns sadly for his old home.

Design and layout

The very small characters are in the lower left-hand side, looking powerless, while the buildings, made salient by their size, are on the right. Visually, the right-hand side can be understood as what is ‘new’ in the unfolding narrative.

The words describing the house are on the left. The picture of the house is on the right — do they complement or contrast each other? Why are the houses made to look so big? How do you think it makes Clancy feel?
Text organisation

The written text is also in the lower left-hand side, where the description sits in opposition to the house being described.

Table 3: Multimodal meanings — an analysis of Clancy & Millie and the Very Fine House


Working with multimodal texts provides opportunities to develop our learners’ love and interest in literature, enhance their knowledge of visual and written language, as well as develop their visual literacy skills as part of engaging with and enjoying the rich visual culture we live and learn in.

Dr Jon Callow
May 2012

About the author

Dr Jon Callow is an experienced educator and teacher, having worked in primary schools, universities and in professional development for teachers. He currently teaches at the University of Sydney, in the areas of English and multiliteracies and he has worked alongside teachers in their classrooms, both in Australia and the United States. His research interests include visual literacy, children’s literature, social justice and pedagogy and digital media.


How this content relates to AITSL teacher standards

Standard 2: Know the content and how to teach it

  • 2.1.1 Graduate Content and teaching strategies of the teaching area. Demonstrate knowledge and understanding of the concepts, substance and structure of the content and teaching strategies of the teaching area.

AITSL Illustration of Practice: Developing detailed content knowledge of subject area — text detail and analysis in English

  • 2.1.2 Proficient Content and teaching strategies of the teaching area. Apply knowledge of the content and teaching strategies of the teaching area to develop engaging teaching activities.

AITSL Illustration of Practice: Using a Dictagloss to support EAL/D students

AITSL Illustration of Practice: Building the field in science to assist students to make connections

AITSL Illustration of Practice: Developing social media profiles to build and represent content knowledge in geography

  • 2.2.1 Graduate Content selection and organisation. Organise content into an effective learning and teaching sequence.
  • 2.2.2 Proficient Content selection and organisation. Organise content into coherent, well-sequenced learning and teaching programs.

AITSL Illustration of Practice: Developing a unit of work for filmmaking

  • 2.3.1 Graduate Curriculum, assessment and reporting.Use curriculum, assessment and reporting knowledge to design learning sequences and lesson plans
  • 2.3.2 Proficient Curriculum, assessment and reporting. Design and implement learning and teaching programs using knowledge of curriculum, assessment and reporting requirements.

AITSL Illustration of Practice: Using images as a summative assessment tool to synthesise learning

  • 2.5.1 Graduate Literacy and numeracy strategies. Know and understand literacy and numeracy teaching strategies and their application in teaching areas.
  • 2.5.2 Proficient Literacy and numeracy strategies. Apply knowledge and understanding of effective teaching strategies to support students’ literacy and numeracy achievement.

AITSL Illustration of Practice: Improving Sentence Structure knowledge using oral language in Year 1

AITSL Illustration of Practice: Using storyboards to develop multimodal texts

AITSL Illustration of Practice: Achieving multiple literacy outcomes through developing and composing multimodal texts

AITSL Illustration of Practice: Developing early literacy through explicit connections between meaning in text, oral language and image

  • 2.6.1 Graduate Information and Communication Technology (ICT). Implement teaching strategies for using ICT to expand curriculum learning opportunities for students.

AITSL Illustration of Practice: Responding and composing using digital texts to support content knowledge in Science Years 5 and 6

AITSL Illustration of Practice: Using ICT to develop student presentations to demonstrate scientific understanding

AITSL Illustration of Practice: Using digital tools to investigate our Indigenous nation

  • 2.6.2 Proficient Information and Communication Technology (ICT). Use effective teaching strategies to integrate ICT into learning and teaching programs to make selected content relevant and meaningful.

AITSL Illustration of practice: Using ICT to create and present multimodal texts in English

AITSL Illustration of Practice: Using ICT to develop social media profiles to develop content knowledge and demonstrate understanding in Geography

AITSL Illustration of Practice: Developing peer assessment through peer evaluation of digital presentations in Science

Standard 3: Plan for and implement effective teaching and learning

  • 3.3.1 Graduate Use teaching strategies. Include a range of teaching strategies.
  • 3.3.2 Proficient Use teaching strategies. Select and use relevant teaching strategies to develop knowledge, skills, problem solving and critical and creative thinking.

Standard 6: Engage in professional learning

  • 6.2.1 Graduate Engage in professional learning and improve practice. Understand the relevant and appropriate sources of professional learning for teachers.

Illustration of Practice: Using professional learning to improve teaching with ICT resources

  • 6.2.2 Proficient Engage in professional learning and improve practice. Participate in learning to update knowledge and practice, targeted to professional needs and school and/or system priorities.