Khan Academy: Learning Habits vs. Content Delivery in STEM Education

Co-written by David Castillo and Peter McIntosh
Most math education analyses in urban high school classrooms focus on delivery of content: What content to deliver, when to deliver it, how to explain it, what textbooks to use, how much home work to assign, and more. As reform efforts have shifted to technology and online learning, we are still asking questions about content delivery in a different context: How to automate it, individualize it, manage it remotely, and deliver it without killing trees.

Improving content delivery helped, but not enough

Oakland Unity High School is a four-year (grades 9-12) public charter high school located in the tough urban neighborhood of East Oakland. Approximately 95 percent of students are Latino or African-American and 85 percent receive free or reduced lunch.
In the summer of 2010, we conducted a diagnostic test with all incoming freshman to evaluate basic algebra and arithmetic skills. The results were astounding; most students needed to retake an Algebra Readiness or Algebra 1 course. In doing so we saw an improvement in proficiency scores, but it wasn’t enough.
The number of students scoring below basic (approximately score of 40 percent) decreased from 77 percent to 28 percent.  The number scoring above proficient (approximately score of 60 percent) increased from 9 percent to 32 percent. We decided to then dive further into what was causing low student performance.
By any reasonable criteria none of the answers to the old questions worked. At Oakland Unity High School in Oakland, Calif. we decided it was time to ask a new question: Are the academic and personal habits of students at the root of performance obstacles in math education?

Poor learning habits revealed the core problem

Through close observation and student interviews, we found that students failed to engage in the coursework and spent little to no time studying. Students were disengaged from their learning responsibilities and the derailing of their studying began as early as elementary school.
Early on, students struggled and faced constant frustration on a series of topics: long division, fractions, negative numbers, and word problems. This discouraged students and led to a decreased interest in engaging in the classroom or completing homework. Yet the bar of expectation and demonstration of competency was low. Students still continued on to the next grade in a one-size-fits-all educational approach despite their lack of knowledge and understanding around basic algebraic concepts.
We concluded that the real problem was making those poor habits an excuse for the wrong initiatives. Instead, we should make those bad habits the target of better initiatives. We then looked to Khan Academy as a popular solution worth testing.

Khan Academy offered solutions with real results

In September 2011, we implemented a rotation blended learning model with a Khan Academy learning lab for all students enrolled in an Algebra Readiness or Algebra 1 course to close a “learning and confidence” gap with the students.
When we stopped worrying about whether Khan Academy videos were better than our own lectures or whether the exercises had the appropriate mix of concept vs. drill, we recognized that we had found a powerful tool that reached students and changed their habits in ways we had never even considered possible.
As we studied student interaction with Khan Academy, we began to see important changes in how students reacted to mathematical challenges.

Early results proved encouraging

With the help of Khan Academy, we found early results very encouraging. This year’s Algebra Readiness and Algebra 1 students scored consistently higher on solving equations, absolute value and the first semester final exams. That margin grew substantially for the most rigorous test on systems of equations (from 37 percent to 74 percent). This suggested that improved habits through the Khan Academy approach were creating real improvements among students. This evidence of superior performance was reinforced by the increased portion of scores above 80 percent on all of the tests.

Adapting study habits made the difference

Many Khan Academy features increase the quality and quantity of practice work among our students:

  • Most exercises are not multiple-choice, which eliminates guessing
  • Questions are randomly generated, which eliminates copying
  • The short video clips engaged students and allowed them to replay the material until they understood it; and
  • The online environment and Khan Academy’s overall design appeals to the students, resulting in significant engagement time.

Students completed the Khan Academy exercises, finished written homework, paid attention in class and gained confidence to approach challenging problems. We truly believe that the Khan Academy approach met student’s learning needs in order to deliver real learning in math proficiency.

Snapshot of Unity High School

By most measures we have had great success. Our API (Academic Performance Index) has grown steadily since 2007, from 595 to 735 this past year. Nearly 90 percent of students regularly pass the CAHSEE (Mathematics) in sophomore year, with 70 percent of students scoring at or above the proficient level of 380.
By these traditional measures we are the highest performing four-year public high school in Oakland. Several other schools perform higher, but have the advantage of starting with students in middle school or even elementary school. Nearly 85 percent of our incoming freshmen attended Oakland public schools.
More than 90 percent of students go on to college: 70 percent to a four-year college and 90 percent will be the first in their families to attend college.

David Castillo is the Principal at Oakland Unity High School where he researches, promotes and leads the implementation of technological tools that provide real-time data and individualized learning programs. 
Peter McIntosh, a math teacher at Oakland Unity High School, has worked at the Oakland pubic schools for four years. Prior to teaching, he was a consulting partner for PriceWaterhous and later an Executive Vice-President of Operations and Call Centers for Charles Schwab.
Follow David and Peter on Twitter at @OaklandUnityHS.

Guest Author

Getting Smart loves its varied and ranging staff of guest contributors. From edleaders, educators and students to business leaders, tech experts and researchers we are committed to finding diverse voices that highlight the cutting edge of learning.

Discover the latest in learning innovations

Sign up for our weekly newsletter.




Michael Round

Well done! Exciting to read about stories like this.

Andrea Phelps

I know that when I study math using the Khan Academy I totally "GET IT" something that really never happened before to me in my math career

Pankaj Parekh

Great reading. I would love to figure how can we participate in experiments like this in our school in Fremont, CA

Chris Gonzales

That's for writing this Mr. Castillo. I'm hoping to implement this for a similar kind of school near Seattle, Washington. This gives me tremendous insight to bring to the discussion table. I have some other questions for you. How can I best reach you?

Joan Noyan

What an achievement. However, I'm not really surprised as I have found the maths sessions a fantastic learning tool. It is engaging and encouraging. Great.

Thank you students and teachers a very good job.

Great job everyone what difference correct equipment and commitments makes.

Wan Park

Glad to hear that it's working for your school. I want to know more about the students' progress in applying their learned skills to solve complex problems. Studies suggest that learning sub-skills separately tend to prevent students from integrating those skills without learning through a whole-task approach. Also, what do they do offline in class? Do they get enough problem workshop time to learn from each other? Did your teachers go through extensive training in implementing the "flipped class model?"

John Bennett

I'm a firm believer that with the necessary skills, the core factual knowledge will be used - and thus understood and learned! With only the core factual knowledge, it's most often short term learned (memorized) - maybe enough for standardized tests, probably not.
The flipped classroom with use of Khan Academy is obviously working here AND should be considered more broadly.

Lärar Saskia

Well done! I'd like to know more about how you use Khan Academy. Did you use i exclusively or with other materials as well? Did you flip the classroom? What about hardware? What did your students use? Are you in a 1-1 program? And so on! I'm a teacher in Sweden who's looking into KA and have also tried it för a bit with my students. We have the disadvantage of not beeing native in english but I still think that this is part of the future and if it works for some student then it's wel worth to put some time and effort into it.

David Castillo

Thanks for reading and the important feedback. This link will take you to a recent presentation that should answer some of your questions. May raise new questions.
Our contact info can be found on the final slide. Thanks!

Mark Harrison

Now it all makes sense. Way back, I got A+'s in math and from my experience I found 3 essential elements for that success. First, I could never miss a class or I'd be lost, so no getting sick; same thing with homework.. only by doing all the homework, would I have the prerequisite understanding to fully understand the next class. Finally, when I occasionally got stuck on a problem, I could ask my dad, an electrical engineer. By using Khan for the lectures and using the teachers to help in doing the 'homework', you provide the same 3 key elements to everyone. Kudos. Too bad our powerful teachers unions here in Canada block charter schools and thus innovations like yours.

shakir khan


Leave a Comment

Your email address will not be published. All fields are required.