Does using digital technology improve student outcomes?

Digital technology use in schools is an interesting topic of discussion.  

Students on the whole enjoy using technology.  They do feel frustrated by technology that doesn't work or by limited access to technology. 

There are parents who want their children to learn about and use technology in schools yet there are also many who see the use of technology in schools as more screen time, an obstruction to learning and believe their children should be taught mainly without technology use.

There are teachers who embrace technology in the classroom with eagerness and enthusiasm yet there are also many teachers who struggle to utilise technology effectively as they feel challenged by their own lack of knowledge and skill.

Resourcing technology can be a challenge for many schools.  How do we address equity issues?  Is BYOD a practical answer?  As technology needs to be replaced so often, is the cost worth the benefits?

The Ministry of Education in New Zealand will fully integrate digital technology into the New Zealand Curriculum by 2018.  

"Formally integrating digital technology into the curriculum is intended to support young people to develop skills, confidence and interest in digital technologies and lead them to opportunities across the IT sector." 
https://www.education.govt.nz/news/digital-technology-to-become-part-of-the-new-zealand-curriculum-and-te-marautanga-o-aotearoa/ 

Click on the image to find out more from the MOE

Click on the image to learn more about the vision for New Zealand education by 2025 

So the question is

Does using digital technology improve student outcomes?

John Hattie’s 2009 research finding for computer enhanced learning

In 2009, Professor John Hattie identified computer assisted instruction as having a .37 effect size when looking across 76 meta-analyses involving 4,498 studies on computer assisted instruction.  This was part of a synthesis of over 800 meta-analysis relating to learning and achievement (Visible Learning) which ranked 139 influences on learning.  Hattie (2009) used the average effect size of .40 as the “hinge point” to determine the success of influences on achievement supporting the claim that with an effect size of .37 computer enhanced learning is less effective than other learning strategies (Waack, n.d.).

It is important to note that Hattie (2009) identified variability across the meta-analyses and that the term ‘computer’ covers various meanings and implementations.  The Visible Learning study highlighted that many of the studies compared classroom teaching with and without computers rather than students learning in different ways using computers.  The analysis emphasised that within the wide range of studies focussed on using computers in the classroom, effective use occurs when teaching strategies are diverse, there is support and training before implementation, opportunities are provided for students to practice, students are in control of their learning, optimal peer learning is utilised and feedback is optimised.  This analysis supports Hattie’s assertion that while computers can increase the probability of learning, there is no necessary relation between using computers and learning outcomes.

Measuring the impact of one-to-one laptop use in literacy for primary age students

Researchers in the studies referenced below used a range of data to consider the impact of one to one laptop use including student achievement data, student and teacher surveys (including information on access and usage) and classroom observations.  When analysing achievement data, many studies (Bebell, & Kay, 2010; Lowther, Inan, Strahl, & Ross, 2012; Shapley, Sheehan, Maloney, & Caranikas-Walker, 2011; Zheng, Warschauer, & Farkas, 2013) explained that when assessing student achievement through standardised tests, there is moderate positive shift as a result of implementing one-to-one laptops.  Many of these studies (Cavanaugh, Dawson, & Ritzhaupt, 2011; Lowther, et.al., 2012; Shapley, et. Al., 2011; Warschauer, M., & Matuchniak, T. 2010; Zheng, et. al., 2013) have raised the question of whether the skills and knowledge gained from using one-to-one laptops is measurable through standardised testing as 21^st Century skills involve creativity, innovation, collaboration, adaptability, self direction, social skills and responsibility which are currently not well represented within traditional school assessment.  Sources of data other than achievement data identify that by using one-to-one laptops, students gain technical expertise and behavioural challenges are reduced with the conclusion that while these findings have not improved standardised testing results, they have a positive effect on the classroom and future academic and career prospects (Shapley, et. Al., 2011).  Positive classroom behaviour, use of student-centred teaching strategies (e.g. student inquiry and lessons that directly support the development of critical thinking, communication, collaboration, and creativity), and more frequent laptop use for active learning resulted in stronger engagement in learning (Lowther, et.al., 2012; Shapley, et. Al., 2011;). 

When considering the impact of one-to-one laptops on students with disabilities, studies by Cowley, (2013) and Corn, Tagsold, & Argueta, (2012) identified that laptop use levelled the playing field for students with disabilities by increasing access to the curriculum, improved assessment methods, allowing the students to receive immediate feedback, promoting positive social interaction, enhancing organisation, enabling them to practice the curriculum at their level and increasing their capability and confidence of successfully completing the task.

Writing is a curriculum focus that has shown achievement gains using one-to-one laptops (Jesson, McNaughton, & Wilson, 2015; Zheng, et. al., 2013; Zheng, Warschauer, Lin, & Chang, 2016).  Student and teacher surveys and interviews from these studies identified that students using laptops were engaged, developed a better understanding of why and how to write for a specific audience and learnt to take responsibility for their own learning.  Warschauer, Zheng, Niiya, Cotten, & Farkas, (2014) highlight the benefit of laptops when writing for at risk learners who used the laptops more often than their peers and for a variety of learning purposes, practising their communication skills through online writing and discussion. Of particular interest, in a study by Bebell, & Kay (2010) was the finding that students completing a written essay assessment on their computers wrote longer and scored higher compared to students responding to the same prompt using pen and paper.

One study that reported positive impact on reading (Rosen, & Beck-Hill, 2012), implemented a constructivist technology enriched programme (one-to-one) where it was noted the experimental group were observed providing independent learning, intellectual challenge, teacher modelling, instructional adjustment and feedback considerably more often than the control group.  Rosen and Beck-Hill (2012) assert that this programme significantly increased learning achievement and provided differentiated instruction where there was increased teacher-student interaction, a range of teaching models, frequent and complex examples of differentiation, more student collaboration and higher student engagement than in more traditional classrooms.

Comparing recent research to John Hattie’s 2009 research finding

When considering John Hattie’s Visible Learning (2009) research finding, it is interesting to note that his updated 2015 effect size for computer assisted instruction is now .45 and above the “hinge point” of .40 identified in his 2009 work.  This indicates that computer assisted instruction is now seen as a stronger influence on achievement than in 2009.

The meta-analysis and research synthesis focused on learning in one-to-one laptop environments by Zheng, et. al., (2016) emphasised that current research on this topic is not doing a good job of assessing student outcomes that are not well captured by current standardised testing.  Zheng, et. al. (2016) identified almost all of the 70 studies reported increased academic achievement yet some also highlighted that technology alone could not change pedagogy.  They further identified that use of laptops make certain outcomes likely (e.g. drafting, editing and sharing writing).  These assertions support my belief in which substituting paper and pen with a laptop does not automatically equate to changed or enhanced learning experiences yet if teachers change the way they construct lessons and support learners, laptops can be a catalyst for 21^st century learning.

So yes I do believe digital technology can improve student outcomes when accompanied by teacher practice that provides effective instruction.  This is where there is increased teacher-student interaction, a range of teaching models, frequent and complex examples of differentiation, more student collaboration and higher student engagement.  Digital technology use is effective for learning where teaching strategies are diverse, there is support and training before digital implementation, opportunities are provided for students to practice, students are in control of their learning, optimal peer learning is utilised and feedback is optimised.

21^st Century skills involve creativity, innovation, collaboration, adaptability, self direction, social skills and responsibility which are currently not well represented within traditional school assessments yet the effective use of digital technology in classrooms will allow learners to develop and refine these skills.

The questions we now need to explore are:

How do we support our whanau and community to understand and value the use of digital technology in our classrooms?

How do we support our teachers to further develop their own skills in using digital technology effectively in the classroom?

How do we address issues of equity and access to digital technology?

References

Bebell, D., & Kay, R. (2010). One to one computing: A summary of the quantitative results from the Berkshire Wireless Learning Initiative. The Journal of Technology, Learning and Assessment, 9(2).

Cavanaugh, C., Dawson, K., & Ritzhaupt, A. (2011). An evaluation of the conditions, processes, and consequences of laptop computing in K-12 classrooms. Journal of Educational Computing Research, 45(3), 359-378.

Corn, J., Tagsold, J. T., & Argueta, R. (2012). Students with special needs and 1: 1 computing: a teacher's perspective. Journal of Research in Special Educational Needs, 12(4), 217-223.

Cowley, B. J. (2013). The Effects of One-to-One Computing for Students with Disabilities in an Inclusive Language Arts Class. ProQuest LLC. 789 East Eisenhower Parkway, PO Box 1346, Ann Arbor, MI 48106.

Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.

Jesson, R., McNaughton, S., & Wilson, A. (2015). Raising literacy levels using digital learning: A design-based approach in New Zealand. Curriculum Journal, 26(2), 198-223.

Lowther, D. L., Inan, F. A., Strahl, J. D., & Ross, S. M. (2012). Do one-to-one initiatives bridge the way to 21st century knowledge and skills?.   Journal of Educational Computing Research, 46(1), 1-30.

Rosen, Y., & Beck-Hill, D. (2012). Intertwining digital content and a one-to-one laptop environment in teaching and learning: Lessons from the time to know program. Journal of Research on Technology in Education, 44(3), 225-241.

Shapley, K., Sheehan, D., Maloney, C., & Caranikas-Walker, F. (2011). Effects of technology immersion on middle school students’ learning opportunities and achievement. The Journal of Educational Research, 104(5), 299-315.

Waack, S. (n.d.). Hattie Ranking: 195 Influences and Effect Sizes Related To Student Achievement. [Blog post]. Retrieved from https://visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/

Warschauer, M., & Matuchniak, T. (2010). New technology and digital worlds: Analyzing evidence of equity in access, use, and outcomes. Review of Research in Education, 34(1), 179-225.

Warschauer, M., Zheng, B., Niiya, M., Cotten, S., & Farkas, G. (2014). Balancing the one-to-one equation: Equity and access in three laptop programs. Equity & Excellence in Education, 47(1), 46-62.

Zheng, B., Warschauer, M., & Farkas, G. (2013). Digital writing and diversity: The effects of school laptop programs on literacy processes and outcomes. Journal of Educational Computing Research, 48(3), 267-299.

Zheng, B., Warschauer, M., Lin, C. H., & Chang, C. (2016). Learning in one-to-one laptop environments: A meta-analysis and research synthesis. Review of Educational Research, 86(4), 1052-1084.