PETAA PAPER 209

Understanding research and evidence: a guide for teachers

Susan Feez and Robyn Cox

Evidence-based teaching is teaching in which decisions made by the teacher are informed by research evidence. There is, however, much debate about what counts as evidence, and how teachers should apply evidence to their practice. This PETAA paper provides teachers with knowledge they can use to:

• reflect on what counts as evidence in relation to English education
• evaluate the strength of evidence derived from different research methodologies to their own practice
• reassess their current ideas about how to implement the English curriculum
• extend their interpretations of current debates in relation to research evidence

Teachers have become accustomed to enthusiasts promoting new approaches they claim will lead to gains in student achievement. For this reason, teachers need to evaluate critically claims that a particular practice is ‘evidence-based’ to ensure that these claims are not merely contributing to the apparently endless stream of novel trends that are routinely visited on teachers and their students.

Evidence-based teaching is typically promoted by those who argue that teaching programs should be based on evidence derived from research designed using scientific experimental methods (for example, double-blind studies and randomised controlled trials) in order to generate reliable evidence that can be used to improve decision-making and practice (Hempenstall, 2006; Wyatt-Smith & Gunn, 2007).

The value of this type of evidence for teachers, however, has been called into question by those who argue that experimental methods designed to answer questions about natural processes are less valid and reliable when applied to cultural and social processes such as teaching and learning in contexts where double blind studies, random selection of participants and control of variables are rarely possible, unless they are used to measure the development of limited, or constrained, sets of skills or knowledge which can be learnt relatively quickly (Freebody, 2007, 2010; Paris & Luo, 2010; Taber, 2013).

A consensus, nevertheless, is emerging among educators that ‘a sound basis for action is converging evidence from multiple sources and different perspectives’ because ‘[e]vidence-based convergence lends strength to findings as no single study, methodology, finding or view is considered, in and of itself, a sufficient basis for action’ (Wyatt-Smith, Elkins, & Gunn, 2011, page 3). This view is supported by Myhill, who argues that: ‘the most robust evidence comes from an accumulation of studies, preferably including replication studies conducted by others’ (Myhill, 2016, page 321).

The term evidence is used routinely, but not always precisely, to promote particular teaching practices, alongside terms such as research, data and methodology. These terms are explored below.

What is evidence?

The word evidence has its origin in a Latin word which means ‘to see’, the same origin as words such as visible and vision. The e- at the beginning of the word is from the Latin prefix, ex-, meaning ‘out of’. So, evidence emerges out of what we can see. Because evidence is something that has been seen, and so verified, it can be used to support a claim or assertion that something is true.

In the legal system, during a criminal trial in a court of law, lawyers for the prosecution will present evidence, perhaps physical evidence gathered by investigators or evidence provided by witnesses, to support their case that a convicted person, the defendant, is guilty of a crime. Lawyers representing the defendant will produce evidence to support the opposite view. The task of the trial judge and jury is to weigh up the evidence before coming to a decision about the guilt or otherwise of the defendant. A comparable task falls to teachers when they are asked to weigh up the evidence before deciding whether to apply a particular practice in their classrooms.

In the field of science, scientists answer their questions and test their theories about natural phenomena by gathering empirical evidence. Empirical evidence is evidence that can be perceived by the senses. It is made up of data collected through observation, experiment and/or measurement. The data are interpreted through statistical analysis, and evidence obtained in this way can be further tested, and ideally, reproduced. Techniques for collecting, analysing and interpreting data to produce evidence vary from one field of academic inquiry to the next. Part of the task for teachers when weighing up evidence is to find out which techniques from which field of academic inquiry were used to analyse the data, and to make judgements about the relevance of these research techniques to their teaching situation.

What are data?

The word data is the plural form of the Latin word, datum, which means ‘something given’. In a scientific study, each piece of information, for instance each observation or each measurement, is one datum. When these pieces of information are collected together, they are called data, a word which in academic English is used as a plural noun indicating that each piece of information can be counted. In everyday English, however, the word data is commonly used as a singular noun, referring to information as a single ‘mass’ rather than as separate countable items.

In controlled experiments, data are collected by measuring, observing and/or recording some elements that can change (variables), while other potential variables are controlled so they do not change. Data collection can be undertaken by participants in the setting or activity under investigation, or by observers looking on, who are not participants.

When patterns are found in the data, and these are interpreted, data become information. Information organised in an abstract, systematic way so that it can be generalised, transferred and/ or re-applied productively in other contexts becomes evidence.

It is interesting for teachers to consider which elements of their practice and of student progress can be counted or measured in some way, and which cannot. School attendance and test scores, for example, can be counted or measured, and expressed as quantitative data. Records of classroom observation, lesson study, student performance or interviews with students and parents, perhaps captured in the form of notes, photographs, audio-visual recordings or work samples, are qualitative data. This leads to questions about how teaching practice and student progress that cannot be easily measured or recorded might be evaluated.

What is research?

Research, according to Dörnyei (2007, page 15), simply means trying to find answers to questions’.

Researchers can find answers to their questions by undertaking an original investigation (primary research) or by searching for what other researchers have already found (secondary research).

Often an investigation begins with secondary research, such as literature review, to find out what is already known and where the gaps are in our knowledge, followed by primary research to gather data that will help answer questions to fill the knowledge gaps. Primary research can be an experimental and/or theoretical investigation to build new knowledge (pure basic research), to solve a practical problem (strategic basic research), or to build knowledge for a specific application or objective (applied research).

Most educational research is applied research, undertaken to improve learning outcomes for students. Applied research in education draws on theories from a range of academic disciplines, for example, psychology, sociology, linguistics and history. What counts as evidence, and how it is collected, analysed and interpreted, varies from one field of study to the next. This presents a challenge for teachers.

What is methodology?

A methodology is a systematic and principled approach, or framework, used by a researcher to design a study to answer research questions. This approach will usually include a theoretical framework and a set of techniques and methods, which can be either quantitative or qualitative. Quantitative and qualitative methods can be combined in mixed or multi method studies.

(1) The word ‘statistics’ comes from the Latin word status, which originally meant a place or arrangement. Statistics is the mathematics ‘of the state’ used to analyse large amounts of quantitative data to identify values that might be significant for planning or policy development.

Quantitative techniques are used to gather data, potentially very large quantities of data, which can be measured and analysed using statistics (1). In the sciences, this is achieved by designing experiments under laboratory conditions so variables can be controlled, the experiment can be repeated, and the findings can be generalised. Randomised controlled trials (RCT) are sometimes referred to as the ‘gold standard’ of research design. In the social sciences, quantitative research often involves the statistical analysis of large-scale survey responses and test results.

Qualitative techniques are used to gather data through observation, record-keeping and by interpreting what people participating in the research do or say. Because qualitative research investigates human experience, it is much more difficult to control variables. This problem can be overcome to some extent by using techniques also used to collect quantitative data, in which participants are randomly assigned to separate intervention and control groups, and by ensuring the study is double blind, that is neither the researcher nor the participants know who is in which group while the intervention is happening. If the research is being undertaken with one particular group, for example, a case study of students in one classroom, the study is considered to be quasi-experimental, and the findings not as reliable. Methodologies, research and data collection techniques used in some exemplar English language education studies are listed in Table 1 below.

What counts as evidence?

The first step is to consider whether the evidence is strong or weak. If a teacher tries out a strategy or implements a program once or twice, and the students are engaged and demonstrate improvement in their learning, the teacher is likely to share their personal experience of the success of that strategy or program with colleagues. Anecdotal evidence of this type, however, is considered to be weak.

While it is always worth learning from our colleagues’ personal accounts of successful professional practice, we cannot be sure how or why the practice was successful, or whether successful implementation of the practice could be repeated with different students by a different teacher at a different time.

Case studies

To examine more closely the potential effectiveness of a teaching practice in an actual teaching situation with a specific group of students, the implementation of the practice might be carefully and systematically recorded and described in detail, from the planning stage to final assessment of student achievement. A case study of this type often begins with a systematic description of the wider context in which the practice is being implemented, for example, an ethnographic description, which includes the cultural and social circumstances of the students and their families, the curriculum context and the teaching situation, the age of the students, their stage of development and their learning needs.

In educational research, the ‘case’ being studied might be a particular teaching practice or program, a teaching intervention, the progress made by a single student, one whole class or school of a particular profile (a single case), or a group of students or multiple classes or schools of comparable or diverse profiles (a multiple case).

A case study can include the use of quantitative and qualitative methods, and can be used to illustrate, evaluate, explain and/or explore a teaching practice or program of interest in a particular context. A case study can also be used as a pilot study in preparation for undertaking a study on a much larger scale (Mills, Durepos, & Wiebe, 2010; Yin, 2014).

Case study evidence is not considered to be strong evidence because it is not based on data drawn from a large population, and so cannot be used for statistical generalisation. Nevertheless, evidence based on a case studied in context has the potential to be transferred to other comparable contexts (Jensen, 2008). Case study evidence can also be used to challenge generalisations based on evidence emerging from large-scale experimental studies, and to question whether a general finding is applicable in specific contexts of interest.

Standardised testing

Evidence drawn from standardised testing of large student populations is considered to be strong evidence. In Australia, the National Assessment Program: Literacy and Numeracy (NAPLAN) comprises a collection of standardised tests designed to assess whether Australian students are achieving minimum outcomes in foundation skills of reading, writing, language and numeracy at key stages of schooling. So students are tested as consistently as possible, all aspects of the delivery of the test are standardised, including the design of the test items, the conditions under which students sit the tests, and the way student responses are scored and interpreted. NAPLAN results provide a snapshot of what each individual child, each class of students, each year group, school, sector, system or region, and the national cohort, can achieve on one day when tested for a limited set of skills and knowledge. Because the test is standardised, student performance at each of these levels, and across time, can be compared. The findings can be used by parents, teachers, schools, systems and funding providers to track progress and to identify where intervention and/or extra resources might be needed. NAPLAN is also promoted as a means for making teachers and schools more accountable.

Although NAPLAN tests are standardised and NAPLAN data are collected from a large population at regular intervals, the statistical reliability of the data, and comparability of the results from one year to the next, and from one individual, school or sector to the next, have been questioned. In other words, despite NAPLAN being a large-scale standardised test, there are differing opinions about the strength of the evidence supplied by NAPLAN, and how much it can be relied upon to guide decisions about teaching practice. Moreover, because NAPLAN is a single snapshot of a limited range of skills, it does not reflect student achievement across all areas of the curriculum, which raises questions about the validity of some of the ways NAPLAN data are used (for example, Wu, 2016). While the purpose of NAPLAN is to improve teaching and learning, because NAPLAN is becoming increasingly high stakes for teachers and schools, ‘much teaching is now aimed at improving NAPLAN scores’ (Lingard, et al. 2016, page 6), with the potential perverse effect of narrowing the curriculum and reducing the capacity of teachers and schools for differentiating the curriculum to meet the diverse learning needs of students.

Experimental methods

Evidence that is statistically the strongest is based on data collected using experimental methods such as randomised controlled trials. In this type of experiment participants are assigned randomly to separate test and control groups. All variables are controlled, and kept constant, with the exception of one, which, in the test group, is deliberately manipulated. The responses of participants in both groups are measured. Data collected from the two groups are compared using statistical analysis during which the effect size of manipulating the one variable is calculated for significance. Experiments of this type are designed so they can be repeated in exactly the same way, with the potential for strengthening the evidence even more. Experimental findings emerging from randomised controlled trials are promoted as delivering the strongest evidence for the effectiveness, or not, of a particular teaching strategy or practice (Kamil, 2012) because the evidence is considered to be generalisable.

The evidence becomes even stronger when results from multiple studies of a similar or related type are collected and collated. This is achieved when researchers undertake a systematic review or meta-analysis. A systematic review, for example, a literature review, is a review and synthesis of evidence gathered from a selection of related studies (for example, Chall, 1967; Freebody, 2007; Snow, Burns, & Griffin, 1998). A meta-analysis is the use of statistical methods to synthesise the results of related studies, often to compare the effect sizes of different treatments or interventions (for example, Hattie, 2009; Myhill & Watson, 2014).

Teachers as researchers

Assessment

To plan an effective program of teaching and learning, teachers collect information to identify what students already know (baseline assessment), as well as student learning needs and goals (diagnostic assessment). Then, during the implementation of the program, teachers monitor and keep track of student progress (formative assessment), giving feedback and adjusting their teaching as needed before assessing student achievement (summative assessment). Teachers use the evidence they gather from this cycle of assessment to evaluate and reflect on their teaching, and to program for subsequent teaching (Centre for Educational Statistics and Evaluation, 2015; Di Gregorio, 2016).

Teachers undertake these different types of assessment by drawing on a repertoire of assessment strategies. Depending on the assessment strategy used, the information collected can be represented as quantitative data (such as test scores, numerical marks, percentages, rankings) or as qualitative data (such as observation notes, annotated work samples, reports, portfolios). Teachers use these data to inform their programming decisions, to provide students with targeted feedback, and for reporting. So, in a sense, teachers who integrate assessment effectively into their program are already using evidence to inform their decision-making. In other words, they are already using evidence-based research to shape their practice.

For classroom assessment to be principled and ethical, it needs to be designed with some key features. First, the assessment must be transparent, so students are clear beforehand about what is expected of them and what is to be assessed; and practical, so that the burden on teachers and students in terms of time, effort and stress is proportionate to the educational benefit derived from collecting the information. Second, the assessment must be valid, that is, it assesses what it claims to assess, and the data collected is used in meaningful ways, and not to support unrelated or over-stated claims. At the same time, assessment must be reliable, so that assessment results are as consistent as possible from one student to the next, from one teacher to the next, and from one time period to the next. Finally, assessment tasks should be authentic, and useful in the context of real-world needs and demands, so as students prepare for the assessment, they engage with curriculum content that is meaningful and of value for meeting their long term educational needs and goals.

The principles that apply to the design of classroom assessment also apply to educational research design. These principles can be illustrated in the context of two methodologies for conducting research in classrooms, methodologies that can be considered in some ways as more rigorous and systematic versions of the assessment processes routinely undertaken by teachers in classrooms: action research and design-based research.

Action research

Action research is a form of enquiry teachers can use to understand their work practices and how their work impacts on their students and/or colleagues. Action research is a systematic way for teachers to reflect on their practice, and to investigate questions about how to improve their practice (Burns, 2010). It has been described by Kemmis, McTaggart and Nixon (2014, page 4) as a type of research ‘oriented to making improvements in practices and their settings by the participants themselves’.

Through action research, teachers can evaluate and improve their practice by examining successes to understand how and why they worked and by investigating challenges and problems with the goal of improving learning outcomes. The action research process begins when the teacher formulates a question, for example: When I do X, why does Y improve, but Z does not? How can I improve … ? In order to answer the question, the teacher designs an action research plan. The design of the plan will incorporate:

• informed action, that is, a principled change to the teaching context or teaching practice with the goal of improving learning outcomes
• procedures for collecting data before, during and afterwards
• procedures for reflecting on and interpreting the data to revise the plan and to inform the next action in the cycle

The action research plan must be practical and achievable in the teaching context. The first step in the plan is the collection of baseline data, that is, data collected before any action is taken. The second step is to implement action, recording the effect. The teacher then reflects on what happened in order to revise and plan for the next action in the series. The data collected during each action – evaluation – reflection cycle is called incycle data. Action research findings can be used as evidence to support changes in classroom practice. These changes then need to be monitored to see if they continue to have a positive impact on the learning outcomes of the students in the class over time. The findings can be shared with colleagues, who might trial the same or similar actions, or the findings might be used to generate further questions to investigate.

When teachers undertake action research, they often collaborate with university researchers. University researchers can assist with the design of the action research and systematic data collection, as well as the use of theoretical concepts to analyse and interpret the data and to enhance the strength of the evidence emerging from the project. University researchers can also assist with the publication of findings, so these are shared more widely, and can become a starting point for classroom based innovation and research elsewhere.

Design-based research

Another methodology used for research collaborations between teachers and university researchers is design-based research, involving ‘small, pragmatic, planned and classroom data-informed interventions, designed with the intention of developing theory about and demonstrating evidence of effective literacy pedagogic practice’ (Comber, Freebody, Nixon, Carrington & Morgan, 2016, p. 316). This methodology has been developed to blend ‘empirical educational research with the theory-driven design of learning environments’, and it has been described as ‘an important methodology for understanding how, when and why educational innovations work in practice’, and the relation between theory and practice. The aim of this type of research is both to ‘create usable knowledge’, while at the same time advancing ‘theories of learning and teaching in complex settings’ (The Design-based Research Collective 2003, page 5).

When university researchers and teachers collaborate to improve practice and to generate evidence of effective practice, the design of the project must take into account both the ethics of using students as research participants, and the theory to be applied as a tool for organising, thinking about and interpreting the data.

Research ethics

The ethics of designing classroom-based research echo the principles that apply to the assessment of student learning in classrooms. In other words, classroom-based research must be transparent, practical, valid, reliable and authentic. In addition, researchers must account for the fact that the relationship between students and teachers is not an equal one, so they must ensure the research has ‘merit and integrity’, is justifiable and respectful, of potential benefit, in the best interests of the students, and safe, protecting the students’ physical, emotional and psychological wellbeing (National Statement on Ethical Conduct of Human Research, pages 29-31). Before participating in research, students, and their parents for those under 18, must give their consent. Because educators are in a position of authority, classroom-based research is designed in ways that ensure parental and student consent is informed and freely given. Parents and students need to be assured that there will be no negative consequences if they decline to participate or choose to withdraw at any time.

Theory

In order to organise, think about and interpret data, researchers use theory. In everyday language, we often use the word ‘theory’ when we are talking about possibilities or guesses, for example: ‘In theory, we should be able to win this.’ Researchers, on the other hand, use the term ‘theory’ to label the conceptual framework, body of knowledge or model they use to organise, explain and interpret data. A theory is like a map the researcher uses to find their way around the research context and the data, and to locate different elements in relation to each other. Often research is designed to test the explanatory power of a theory in a specific context, with the result that the findings might modify or extend the theory.

Practitioners often dismiss theory as not relevant to resolving their day-to-day challenges and problems. Nevertheless, working from a sound theory or model enables practitioners to reflect on their practice systematically and critically, and also to generalise on the basis of reflective practice in order to sustain successful innovations over time and to implement innovative ideas emerging from their findings in different contexts.

The idea that good theory and good practice depend on each other has echoed through the centuries, as illustrated in the following maxims, the first by Leonardo da Vinci in the fifteenth century, and the second, in the twentieth century, by Kurt Lewin, who coined the term ‘action research’.

Evidence-based practice in the classroom: debates and challenges

Particular types of instruction are often promoted by educational policy-makers and administrators, and commercial interests, as evidence-based in ways that imply the approach works for all students in all contexts. Even when a particular teaching strategy or practice is promoted as being ‘evidence-based’, however, classroom teachers remain responsible for making professional judgements about whether and how the strategy or practice should be applied to the design of teaching and learning programs for the students in their specific teaching situation.

One of the biggest challenges facing teachers evaluating the effectiveness of a particular teaching practice or strategy is the number of variables in any teaching situation. Educational researchers may be able to control some of these variables in the design of experimental studies used to evaluate a particular teaching approach, but classroom teachers are rarely afforded this luxury (Freebody 2007; See also Fehring & Freebody, 2017; Proctor, Freebody & Brownlee, 20152). This raises questions such as the following.

• Can variables controlled in experimental studies also be controlled in real-world classrooms?
• Can teachers in real-world settings faithfully replicate the evidence-based teaching strategy?
• Can teaching resources used in the experimental studies be made available and relevant, or be replicated accurately in real-world settings?
• Has the evidence collected by researchers to underpin claims that a particular teaching practice or strategy is evidence-based been adjusted to account for diverse real-life classroom settings, for example, through accompanying ethnographic and field descriptions, systematic observation and/or text analysis?

When evidence derived from research designed using experimental methods is tempered by evidence emerging from research designed using ethnographic, case study and text analysis methods, the distinctive features of specific teaching situations and student profiles can be accounted for. Here are some examples.

• Ethnographic studies, which document the social context in which the research takes place, may produce evidence which teachers find more accessible because it more closely reflects their classroom experience (Purcell-Gates, 2011, 2013).
• While case study findings may not represent large populations, and so cannot be used statistically to make generalised claims, the findings of a case study that includes a detailed description of the teaching and learning context can be transferred to comparable contexts (Jensen, 2008).
• When student work samples are analysed using principled frameworks (a theory) for describing consistently the way students respond to the language and literacy demands of the curriculum, the analysis can reveal which aspects of language and literacy development are key to meeting those demands and suggest teaching practices that might contribute to student progress. Frameworks of this type also provide shared tools for assessment, and for comparing the language and literacy demands of different educational contexts, identifying the distinct language and literacy demands of specialised learning areas, such as Science or History, and understanding the way multimodal and digital texts make meaning. (Callow, 2013; Derewianka 2011; Derewianka & Jones, 2016; Humphrey, Droga, & Feez, 2012; Polias, 2016 ; Rose & Martin, 2012).

With the heavy promotion of teaching practices labelled as ‘evidence-based’ by both policy-makers and commercial interests, teachers need to become critical consumers of this type of promotion. They need to apply their professional judgement in ways such as those described above in order to weigh up competing and contradictory claims, and to avoid ‘cherry-picking’ from the evidence to support a particular agenda, or one side of a debate. An area of English teaching in which teachers must use professional judgement to navigate competing research agendas and high-profile debates is the teaching of early literacy.

The experimental evidence is very strong that intensive systematic instruction in how the sounds of English (phonemic awareness) and how they correspond with the letters of the English alphabet (phonics) has a positive effect on the English word reading skills of children in the early years of school (e.g. Buckingham, Wheldall, & Beaman-Wheldall, 2013), and also to some extent prevents later reading difficulties (Ehri, Nunes, Stahl, & Willows, 2001). As a result, this type of instruction is now heavily promoted, with phonics screening tests soon to be mandated in Australia.

When the high-stakes spotlight is on teaching practices designed to teach sound-letter correspondence (code-breaking), there is a risk that less attention is paid to other critical aspects of learning to read, such as building vocabulary and reading for meaning, as well as using and interpreting the meanings socially and culturally (Freebody, 2007). To counteract this risk, teachers need to be mindful when designing literacy programs that the research evidence also shows the following:

• Evidence supporting the effectiveness of intensive phonics instruction is ‘based on achievement measures that typically test letter-sound knowledge and word knowledge in isolation … not on reading whole texts and developing reading strategies and skills in the context of reading’ (Xue & Meisels, 2004, pages 194–195).
• The ability of children to attend to speech units larger than phonemes (e.g. syllables, onset and rime) is a reliable indicator of future reading success (Goswami, 2006), Morphological awareness, awareness of the meaningful parts of words, is also associated with improved reading comprehension (Kirby, Deacon, Bowers, Izenberg, Wade- Woolley, & Parrila, 2012) .
• To learn to read effectively, ‘children need a balanced instructional approach that includes learning to break the code and engaging in meaningful reading and writing’ (Xue & Meisels, 2004, page 222).
• To be successful throughout the school years, students need to be taught explicitly and cumulatively how to meet the advanced literacy demands of each content area (Freebody & Morgan, 2014, Shanahan & Shanahan, 2014).
• Research evidence that emphasises teaching letter-sound correspondence does not account for the ‘differences in the developmental trajectories of different reading skills’ (Paris & Luo, 2010, page 321), and the fact that constrained skills, such as decoding, are learned relatively quickly, while unconstrained skills, such as reading comprehension, develop much more slowly over the years of schooling and beyond (See also Luke & Woods 2008).
• There is a wide array of research findings in relation to early literacy development, yet, as illustrated in Clark (2017), it is easy for researchers, policy makers and teachers working in this area to foreground evidence that supports a strongly held view, while ignoring evidence that might challenge or contradict that view. Clark (2017) surveys a wide array of research findings evaluating the implementation of a Year 1 phonics screening test in England, and, on the basis of this survey, argues that before a similar screening test is implemented in Australia, a more comprehensive review of the evidence is needed.

With a diverse array of research findings such as these, the task facing a teacher of early literacy is to survey the research findings in order to design, ‘a meaningful assemblage of practices’ that make ‘specific and particular sense for each group of students with whom they work’ (Honan, 2004, p. 37). This involves ‘carefully and thoughtfully’ making ‘a series of professional judgements about what and how to teach’ (p. 39). To assemble literacy teaching practices in a principled way, teachers can use the ‘four resources’ model (Honan, 2004, p. 45). The four resources model can be used to organise literacy resources and practices into four categories: resources used to break the code, resources used to comprehend and participate in the meanings of texts, resources for using texts to meet ‘sociocultural expectations’, and resources for analysing texts to reveal and challenge the predispositions and biases of the writer (Freebody 2004, page 7). This model is a conceptual tool teachers can use to fill ‘the gaps in the map of practices they have created’ (Honan, 2004, page 45) and, thus, to design a comprehensive and balanced literacy program.

Teachers also need to apply professional judgement, and account for the specific circumstances and needs of their students, when interpreting data emerging from all types of high stakes testing (Goss, Hunter, Romanes, & Parsonage, 2015; Taber, 2013). They need to consider what each high stakes test is designed to assess, how it is constructed, how test results are used and the effects of testing, both positive and negative, on their students and school community (Lingard, Thompson, & Sellar, 2016). Large scale data, collected through NAPLAN for example, can be useful for tracking national, state and sector trends in the development of narrow sets of skills, and where resources need to be targeted, but are less useful for understanding the educational progress and achievement of individual students and classes and evaluating the professional knowledge and skills of individual teachers. While NAPLAN data have ‘some positive uses’, classroom teachers also have to take into account evidence of potential ‘negative impacts on learning and on student well-being’ (Wyn, Turnbull, & Grimshaw, 2014, page 6). These include narrowing the curriculum, teaching to the test, reduced student motivation to learn, increased teacher and student stress, labelling students on the basis of deficits, a reduced emphasis on equity and social justice, and diminished capacity for teachers to differentiate their teaching to meet specific student needs (Lingard et al., 2016).

For the foreseeable future, teachers will be working in a context in which evidence-based teaching is foregrounded, and data collected through high stakes testing will be used to measure the performance of students, teachers, schools and school systems. This context requires teachers to be critical consumers of what counts as evidence, which means recognising both the strengths and limitations of educational research, high stakes testing, the evidence these produce and the ways this evidence is used (Lingard et al., 2016; Prain 2017; Taber, 2013). Globalisation combined with the advance of information technologies has led to what has been termed the globalised educational reform movement (GERM) (Sahlberg, 2011) and the ‘datafication’ of schooling (Lingard et al, 2016). These developments have been taken up enthusiastically be policy makers, educational administrators and commercial interests. Nevertheless, classroom teachers remain responsible for ensuring research evidence is used for the benefit of their students and their own professional learning.

Conclusion

So, what does this all mean for teachers? Ever since schools and teaching have been a constant in society, teachers and principals have been earnestly seeking the best outcomes for the learners in their care. To achieve these outcomes, the goal of teachers and schools has always been to reflect on and perfect teaching methods. However, in more recent times this process has been accelerated, mandated and now measured.

This paper provides an introduction to evidence-based pedagogy. It has defined key terms and unpacked some key processes underpinning educational research. The paper attempts to provide some critical knowledge and checkpoints for navigating the complex terrain of educational research and at the same time to show how teachers’ classroom practice, and the decisions made by school and system leaders, might be based on research evidence. To make this clearer Table 2 below takes five well-known primary English teaching strategies or practices and links them to some of the research on which these practices are based. While this table is not exhaustive, it shows some of the evidence that suggest these strategies work. As we have argued in this paper, when choosing strategies and practices to use in their classrooms, teachers must also weigh up the research evidence, both its merits and shortcomings, in relation to the complex configuration of variables that together characterise each unique teaching situation.

In conclusion, teachers and school leaders must be acknowledged for the professional knowledge and skills that enable them to navigate the complexities of educational systems every day, at the same time as they also strive to ensure the best educational outcomes for each learner. Increasingly, within this complex terrain, teachers are required to understand the work of educational researchers and ‘what counts as evidence’. The authors of this paper hope that we have provided some guidelines for English teachers as they undertake this demanding, but professionally rewarding, task.

References

• Australian Society for Evidence Based Teaching
• Bruniges, M (2005). An evidence-based approach to teaching and learning. Melbourne: Australian Council of Educational Research.
• Buckingham, J, Wheldall, K., & Beaman-Wheldall, R. (2013) ‘Why Jaydon can’t read: The triumph of ideology over evidence in teaching reading’ (.pdf 2.8 MB). Policy 29(3), 21–32.
• Burns, A (2010). Doing Action Research in English Language Teaching: A Guide for Practitioners.  New York & London:  Routledge
• Callow, J (2013) The shape of text to come: How image and text work. Marrickville, NSW: Primary English Teaching Association Australia (PETAA)
• Centre for Education Statistics and Evaluation (2014). What works best: Evidence-based practices to help improve NSW student performance. Sydney, NSW Department of Education and Communities
• Centre for Educational Statistics and Evaluation (2015). Reassessing assessment. Sydney: NSW Department of Education
• Chall, J. S. (1967) Learning to read: The great debate.  New York: McGraw Hill.
• Clark, M M (Ed.) (2017) Reading the evidence: Synthetic phonics and literacy learning (.pdf 300 kB). Birmingham, UK: Glendale Education
• Comber, B, Freebody, P, Nixon, H, Carrington, V & Morgan, M (2016), New literacy demands in the middle years — learning from design experiments. In H. de Silva Joyce & S. Feez. Exploring Literacies: Theory, Research and Practice  (pages 315-320). London: Palgrave Macmillan
• Condliffe Lagemann, E (2000). An Elusive Science: The Troubling History of Education Research.  Chicago: University of Chicago Press
• Derewianka, B (2011) A new grammar companion for teachers. Newtown, NSW: PETAA
• Derewianka, B & Jones, P (2016) Teaching language in context (2nd edition) Melbourne: Oxford University Press
• de Silva Joyce, H & Feez, S (2015) Exploring literacies: theory, research and practice.  London: Palgrave Macmillan
• Di Gregoria, E (2016) ‘Teacher as researcher: The reflective processes’ (.pdf 941 kB) Professional Educator: The ACE forum for policy, research and practice in education, 15(3), 16-18
• Dörnyei, Z (2007) Research methods in applied linguistics: Qualitative, quantitative and mixed methodologies. Oxford: Oxford University Press
• Edwards-Groves, C & Davidson, C (2017) Becoming a meaning maker: talk and interaction in the dialogic classroom. Newtown NSW: PETAA
• Ehri, L C, Nunes, S R, Willows, D M, Schuster, B V, Yahgoub-Zadeh, Z & Shanahan, T (2001) Phonemic awareness instruction helps children learn to read: Evidence from the National Reading Panel’s meta-analysis. Reading Research Quarterly, 36(3), 250–287
• Fang, Z (2012) Approaches to Developing Content Area Literacies: A Synthesis and a Critique. Journal of Adolescent & Adult Literacy 56(2) 103-108
• Fehring, H & Freebody, P (2017) Assessment into practice: understanding assessment practice to improve students’ literacy learning. In Fehring, H (Ed.) Assessment into practice: Understanding assessment practice to improve students’ literacy learning  (pages 1–7) Newtown, NSW: PETAA
• Freebody, P (2004) Hindsight and foresight: Putting the Four, Roles Model of reading to work in the daily business of teaching. In A Healy and E Honan (Eds) Text next: New resources for literacy learning  (pages 3–17). Newtown, NSW: PETAA.
• Freebody, P (2007) Literacy education in school: Research perspectives from the past, for the future. Camberwell, Victoria: Australian Council for Educational Research
• Freebody, P (2010) Socially responsible literacy education: Toward an ‘organic relation’ to our place and time. In Christie F, and Simpson A, (Eds) Literacy and social responsibility: Multiple perspectives (pages 40–55). London and Oakville, CT: Equinox.
• Freebody, P & Luke, A (2003) Literacy as engaging with new forms of life: The Four Roles Model. In Bull G & M Anstey M (Eds). The literacy lexicon (2nd Edition (pages 51-65) Frenchs Forest, NSW: Prentice Hall
• Freebody, P & Morgan, A M (2014). Curriculum explicit-literacy: Expanding the repertoire. In A M Morgan, B Comber, P Freebody and H Nixon (Eds) Literacy in the middle years. Newtown NSW: PETAA
• Goss, P, Hunter, J, Romanes, D, & Parsonage, H (2015) Targeted teaching: how better use of data can improve student learning (.pdf 5.1 MB) Melbourne: Grattan Institute
• Goswami, U (2006). Phonological awareness and literacy in Encyclopedia of language and linguistics (2nd Edition). Amsterdam: Elsevier, 489–497.
• Hattie, J (2009) Visible learning: a synthesis of over 800 metanalyses relating to achievement, London and New York: Routledge
• Hempenstall, K (2006) ‘What does evidence-based practice in education mean?’ (.pdf 110 kB) Australian Journal of Learning Disabilities 11(2), 83–92
• Hempenstall, K & Buckingham, J (2016) ‘Read About It: Scientific Evidence for Effective Teaching of Reading: Research Report 11’ (.pdf 8.6 MB) St Leonards, NSW: Centre for Independent Studies
• Honan, E (2004) Using the four resources model as a map of possible practices. In A Healy and E Honan (Eds). Text next: New resources for literacy learning, Newtown, NSW: Primary English Teaching Association Australia.
• Humphrey, S, Droga, L and Feez, S (2012) Grammar and meaning. Newtown, NSW: PETAA.
• Jensen, D (2008). Transferability. In Given L M (Ed.) The SAGE encyclopedia of qualitative research methods  (page 887) Thousand Oaks, CA: Sage
• Kamil, M L (2012) Current and historical perspectives on reading research and instruction. In K R Harris, S. Graham, and T Urdan (Eds.) APA Educational Psychology Handbook: Vol. 3. Application to Learning and Teaching (pages 161–188) American Psychological Association
• Kemmis, S, McTaggart, R, & Nixon, R (2014) The Action Research Planner: Doing Critical Participatory Action Research. Springer
• Kirby, J K, Deacon, S, Bowers, P, Izenberg, L, Wade-Woolley, L, & Parrila, R (2012) ‘Children’s morphological awareness and reading ability‘. Reading & Writing, 25(2), 389-410
• Lewin, K (1951) Problems of research in social psychology. In D Cartwright, (Ed.), Field Theory in Social Science: Selected Theoretical Papers (pages 155–169). New York: Harper
• Lingard, B, Thompson, G, & Sellar, S (2016) National testing from an Australian perspective. In B Lingard, G Thompson, and Sellar (Eds.) National Testing in Schools: An Australian assessment  (pages 1–17) London & New York: Routledge
• Luke, A, & Woo  (2008) ’Accountability as testing: Are there lessons about assessment and outcomes to be learnt from No Child Left Behind?’ In Literacy Learning: The Middle Years, 16(3), 11–19
• Matters G (2006) Using Data to Support Learning in Schools: Students, teachers, systems Camberwell, Victoria: Australian Council for Educational Research
• Mills, A, Durepos, G, Wiebe, E (Eds.) (2010) Encyclopedia of Case Study Research. Thousand Oaks, CA: Sage
• Myhill, D (2016) Grammar for writing. In H. de Silva Joyce & S. Feez. Exploring Literacies: Theory, Research and Practice (pages 320–322). London: Palgrave Macmillan
• Myhill, D, & Watson, A (2014) The role of grammar in the writing curriculum: A review of the literature. Child Language Teaching and Therapy, 30(1), 41–62
• National Early Literacy Panel (2008) Developing early literacy:  Report of National Early Literacy Panel.  Washington, DC:  National Institute for Literacy
• Paris, S G, & Luo, S W (2010) ‘Confounded statistical analyses hinder interpretation of the NELP report’ Educational Researcher, 39, 316–322
• Phillips, D C (2005) ‘The Contested Nature of Empirical Educational Research (and Why Philosophy of Education Offers Little Help)’ Journal of Philosophy of Education, 39(4), 577—597
• Polias, J (2016) Apprenticing students into science: Doing, talking & writing scientifically. Melbourne: Lexis Education
• Prain, V (2017) Simplistic advice for teachers on how to teach won’t work. The Conversation  (9th Nov, 2017)
• Proctor, H, Brownlee, P & Freebody, P (Eds.) (2015) Controversies in education: Orthodoxy and heresy in policy and practice, Dordrecht, The Netherlands: Springer Science
• Purcell-Gates, V (2013) ‘Literacy worlds of children of migrant farmworker communities participating in a Migrant Head Start Program’ (.pdf 2.3 MB) Research in the teaching of English Vol. 48 No.1, 68–97
• Rothery, J (1990). ‘Story Writing in the primary school: Assessing narrative type genres’ Unpublished PhD thesis, University of Sydney.
• Rothery, J (1996) Making changes: Developing an educational linguistics. In Hasan, R & Williams, G (Eds.) Literacy in society (pages 86–123). London: Longman.
• Sahlberg, P (2011) ‘Paradoxes of educational improvement: The Finnish experience’ Scottish Educational Review, 43(1)
• Shanahan, C, & Shanahan, T (2014). ‘Does disciplinary literacy have a place in elementary school?’ The Reading Teacher, 67(8), 636-639
• Snow, C E, Burns, M S, & Griffin, P (Eds.) (1998) Preventing reading difficulties in young children (.pdf 216 KB) Washington, DC: National Academy Press.
• Taber, K S (2013) Classroom-based Research and Evidence-based Practice (2nd edition). London: Sage.
• The Design-based Research Collective (2003) ‘Design-based research: An emerging paradigm for educational inquiry.’ Educational Research 32(1), 5–8
• The National Statement on Ethical Conduct in Human Research (2007) (Updated May 2015)
• Wheldall, K, & Beaman, R (2011) Effective instruction for older low-progress readers: Meeting the needs of indigenous students. In Wyatt-Smith, C, Elkins, J and Gunn, J (Eds) Multiple perspectives on difficulties in learning literacy and numeracy (pages 255–273). New York: Springer.
• Wu, M (2016) What national testing can tell us. In B Lingard, G Thompson, & S Sellar (Eds.) National Testing in Schools: An Australian assessment (pages 18–29) London & New York: Routledge
• Wyatt-Smith, C, Elkins, J, & Gunn, S (Eds) (2011) Multiple perspectives on difficulties in learning literacy and numeracy. Dordrecht, The Netherlands: Springer International
• Wyatt-Smith, C, & Gunn, S (2007). ‘Evidence-based research for expert literacy teaching – Paper No. 12’ October 2007. Melbourne: Department of Education and Early Childhood Development
• Wyn, J, Turnbull, M, & Grimshaw, J (2014) The Experience of Education: The impacts of high stakes testing on school students and their families — A Qualitative Study (.pdf 550 kB) The Whitlam Institute within the University of Western Sydney.
• Xue, Y, & Meisels, S J (2004). ‘Early literacy instruction and learning in kindergarten: Evidence from the early childhood longitudinal study – kindergarten class of 1998–1999’ American Educational Research Journal, 41(1), 191–229
• Yin, R K (2014) Case Study Research: Design and Methods (5th edition). Sage Publications

About the authors

Dr Susan Feez is Associate Professor in the School of Education at the University of New England (UNE), Armidale, NSW. She has worked as a classroom teacher of language, literacy and EALD, and now teaches and researches in the fields of English language, literacies and EALD education. Susan is interested in the development of textbooks and teacher handbooks as a means of fast tracking into classrooms educational innovation derived from research. This has been the motivation for a series of educational publications she has contributed to over recent years.

Dr Robyn Cox is currently Associate Professor of Literacy Education at ACU National, Strathfield campus. Prior to this she was Principal Lecturer at the University of Worcester, UK and a member of the executive committee of the United Kingdom Literacy Association (UKLA). Robyn has also held positions at Universities in Australia, Singapore and Brunei Darussalam. She is the author of several international journal articles in the field of literacy research and has been involved in teacher education in four countries over a 20-year period. Robyn is well known for her commitment to the development of a strong professional knowledge base in initial teacher education and remains dedicated to bringing accessible educational research and theory to teacher education students.

How this content relates to AITSL teacher standards

Coming soon. Watch this space for the relevant professional teacher standards for PETAA Paper 209, with links to illustrations of practice videos on the AITSL website

• Professional knowledge
• Professional practice
• Professional engagement