Searching for Efficacy in Implementation on the EdTech Expo Floor

Jason Ribeiro

When it comes to procurement and implementation, if ISTE signaled one thing it would be that  practitioners and researchers need to refine the decision-making and evaluation processes of EdTech.

Attending the ISTE Conference was a truly eye-opening experience (and I don’t use that phrase often). This was my first time attending (and presenting) and I tried to stay away from peer recommendations and blog reviews so that I could take in the event without any preconceived notions. While I noticed what I am sure are overt features of the conference (i.e. its massive size, diverse presentation topics and vendors, etc.) what stuck with me was how quickly the conversation surrounding EdTech implementation throughout districts is changing. While the expected excitement of teachers and companies enveloped social media, internal/external events, and the Pennsylvania Convention Center itself – rich conversations with EdLeaders from around the world yielded a much more tempered response. School leaders are tired of not knowing whether the technologies they are acquiring are: a) the right fit for their organization and b) having a positive impact on student learning. The temptation of a few football fields’ worth of devices had little impact on educational professionals who traveled to ISTE from far and wide for answers, for guidance, and for efficacy.

It seems that districts and their leaders are beginning to realize that while getting the technology into their classrooms may have been (or still is) a great challenge, knowing which products to purchase and how to effectively assess their impact is where they desperately need assistance. As an experiment I asked over a dozen senior leaders what processes they had in place to determine the efficacy of a purchased device or tool. The response was unanimous – none had established practices that helped protect their students, teachers, and the community’s dollars. However, every single one of them was scouring the Exhibit Hall floor and attending presentations trying to find answers. Below are just a few of the ISTE 2015 technology efficacy and leadership highlights.  

In their Leveraging Data, Research and Collaboration to Advance 1:1 Computing Programs poster, Dr. Damian Bebell, Sujoy Chaudhuri, and Shabbi Luthra highlighted how the simple use of educational research tools and data can help school districts more effectively define and measure the effectives of their technology implementations.

While much of the attention at ISTE 2015 was placed on the launch of new product features and software, senior leaders seemed quite fixated on the long-awaited release of the NMC Horizon Report: 2015 K-12 Edition. Presenters Samantha Becker, Larry Johnson, and Keith Krueger touched upon several obstacles related to EdTech adoption (e.g. the negative impact excessive standardized testing is having on technology use in the classroom). The full 52-page report can be accessed here.

On the vendor side, district leaders and investors seemed to be quite taken by data visualization, survey-building, and dashboard tools related to EdTech (e.g. Qualtrics). There was definitely a greater focus on applications that would help refine decision-making and allow district leaders to get a clearer picture of how students are interacting with the technology in their districts. Teacher and student voice are becoming powerful metrics moving forward as districts begin to make more EdTech investments. In discussing EdTech spending and the procurement process itself, non-profit organization Digital Promise spoke at length about their Improving Ed-Tech Purchasing report completed in partnership with the Education Industry Association. The document draws insight from both educators and providers about their triumphs, challenges, and overall experiences with procurement practices. The report states:

Current purchasing practices were designed for print-based resources, not modern technology. The result is that at times, teachers and students don’t end up with the best learning technology tools to meet their needs. We can do more to ensure the promise of personalized learning is fulfilled.

It was refreshing to know that other researchers at ISTE shared my concerns about the lack of efficacy and evidence-based decision-making in school districts’ acquisition of EdTech. Dr. Peter Cohen, Dr. Henry Turner, and Dr. Gina Flanagan explored the new research about what superintendents do to ensure they are gaining a return on their EdTech investment and that expenditures are focused on advancing teaching and learning. The executive summary of their research paper titled, Framing Innovation: How Do Superintendents Gain Acceptance for Large-Scale Technology Initiatives? can be found here. I was honored to follow them with my paper, Cost-Fixated: Technology Acquisition for 746,000 Ontario Students and (from a Canadian perspective) echo many of their concerns about the lack of formal research being used to guide senior leaders’ decision-making. Acting on peer recommendations, blog reviews, and gut instinct are no longer acceptable methods of effective technology procurement. Further, districts need to be far more tenacious when evaluating the impact of their technology investment. I took the fact that dozens of high-level district leaders attended our 8:30am session on the last day of ISTE as a sign that the search for efficacy is very real. My colleagues and I highlighted that educational innovation (through technology) can and will occur when practitioners and researchers work together to refine the decision-making and evaluation processes.  

From a leadership perspective, my guess is that this trend will continue to grow from year to year. As technology’s footing in our classrooms becomes further solidified, districts and their leaders will continue to come to ISTE for answers. It is now the responsibility of vendors, researchers, and conference officials to ensure that dollars aren’t the only things being exchanged at ISTE 2016 in Denver – best practices, scholarship, and formal evidence from companies need to be flowing as well. The era of being enamored by shiny devices has come to a close – it’s time to get to work.

For more check out:


Jason Ribeiro is a Ph.D. student at the University of Calgary in Alberta, Canada specializing in Educational Leadership. Follow Jason on Twitter at @jason_ribeiro.

Guest Author

Getting Smart loves its varied and ranging staff of guest contributors. From edleaders, educators and students to business leaders, tech experts and researchers we are committed to finding diverse voices that highlight the cutting edge of learning.

Discover the latest in learning innovations

Sign up for our weekly newsletter.

1 Comment


We see this a lot when we work with schools - one of our main customers. So much spotlight is put on the EdTech solutions IN the classroom, while back office functions are often behind the times. Sometimes the best way to get a return on investment and improce the learning environment is to make administrative tasks easier and less time-consuming, so that teachers can use their energy to focus on the students. This is one of the reasons AltSchool decided to implement an e-procurement solution (Procurify)
Case Study:

Leave a Comment

Your email address will not be published. All fields are required.