Company   |   Clinically Proven

The Science Behind Qstream

Harvard Medical School ProfessorDeveloped at Harvard Medical School, Qstream uses an interval reinforcement methodology that is scientifically proven to increase knowledge retention from 3 months to 2 years, and changes even ingrained on-the-job performance. Qstream incorporates the “spacing” and “testing” effects into a simple, mobile delivery platform to produce results that have been proven through many rigorous, peer-reviewed clinical trials.

The Forgetting Curve

The assumption that if you learn something it is retained and can be recalled is simply false. This has been proven many times. Forgetting is a natural, physiological occurrence and must be factored into the teaching and learning process. This phenomenon was documented as early as 1885 when German psychologist Hermann Ebbinghaus published his findings about learning.

The forgetting curve describes the dramatic drop off in knowledge retention over time. Studies show that in as little as 30 days, 79% of knowledge is forgotten. It is simply a matter of how the human brain works.

The Spacing Effect

“Spaced education” or interval reinforcement is a proven way to combat the forgetting curve.  The spacing effect indicates that you can significantly increase knowledge retention if you present information and reinforce it over spaced intervals of time. There is a proven neurophysiological basis for the spacing effect. Learning over time enhances memory and the survival of new neurons. It increases the efficiency of the uptake of information and encodes the information so that it is preferentially retained.

The Testing Effect

Testing is not just a dipstick that measures knowledge.  Testing, or retrieval practice, is an active learning process that can dramatically improve knowledge retention when combined with immediate answer feedback. This process is known as the testing effect. Studies comparing the testing effect to passive learning without testing (reading, watching a video) and conceptual mapping (drawing diagrams to relate concepts) show that testing is the most effective approach.

Results

Numerous, peer-reviewed clinical trials have been to conducted to test the effectiveness of spaced education and Qstream. These rigorous clinical trials show that Qstream significantly improves knowledge retention, on-the-job performance, and user engagement.

Spaced education was initially developed and then rigorously studied by Harvard Medical School Associate Professor, Dr. B. Price Kerfoot, as a method to improve the long-term knowledge retention of medical trainees.

In over 10 large randomized trials completed to date, spaced education has been found to:

  • Improve knowledge acquisition
  • Increase long-term knowledge retention (out to 2 years)
  • Change behavior
  • Boost learners’ abilities to accurately self-assess their knowledge.

In addition, spaced education is extremely well-accepted by participants.

The interval reinforcement methodology is content-neutral and can be utilized to learn most anything. The full multimedia capabilities of the Internet can be harnessed to create a rich and effective user experiences.

Published Research Studies

Gyorki DE, Shaw T, Nicholson, J, Baker, C, Pitcher M, Skandarajah A, Segelov E, Mann GB. Improving the impact of didactic resident training with online spaced education. ANZ J Surg. 2013 Jun;83(6):477-80.

Educational programs are frequently developed to improve the knowledge of medical trainees. The impact of a program may be limited if there is no follow-up to reinforce the message. Online Spaced Education (SE) has been developed to address this limitation. This study was performed to assess whether an SE program would improve the impact of a didactic seminar.

Minter RM. Commentary: teaching the teacher–spaced education as a novel approach to teaching interns to teach, Am J Surg. 2013 Jul;206(1):128-9.

Using spaced education to teach interns about teaching skills.

Pernar LI, Corso K, Lipsitz SR, Breen E. Using spaced education to teach interns about teaching skills, Am J Surg. 2013 Jul;206(1):120-7.

Using spaced education to improve interns’ teaching skills is a potentially powerful intervention that improves interns’ enthusiasm for teaching and teaching effectiveness. The changes are mirrored in students’ ratings of interns’ teaching skills and interns’ attitudes toward teaching.

Tulenko, K., Bailey, R. Evaluation of spaced education as a learning methodology for in-service training of health workers in Ethiopia. Knowledge Management & E-Learning, KM&EL. 2013;5(3), 223–233.

This pilot study in Ethiopia demonstrates that the spaced education methodology is acceptable and effective for the acquisition of knowledge in a low-resource context for course participants with a clinical or public health background and moderately reliable Internet access.

Shaw TJ, Pernar LI, Peyre SE, Helfrick JF, Vogelgesang KR, Graydon-Baker E, Chretien Y, Brown EJ, Nicholson JC, Heit JJ, Co JP, Gandhi T. Impact of online education on intern behaviour around joint commission national patient safety goals: a randomised trial. BMJ Qual Saf. 2012;21:819-825.

This randomized trial at two Harvard teaching hospitals showed that the spaced education methodology improved both the self-reported confidence and patient-safety behaviors of resident physicians in a simulated scenario, compared to control residents receiving traditional online training (online slide show followed by a quiz).

Kerfoot BP, Baker H. An online spaced-education game for global continuing medical education: a randomized trial.  Ann Surg. 2012 Jul;256(1):33-8.

Involving 1470 physicians in 63 countries, this randomized trial demonstrated that the spaced education methodology with game mechanics can substantially improve guidelines knowledge and is a well-accepted method of global CME delivery.

Kerfoot BP, Baker H, Pangaro L, Agarwal K, Taffett G, Mechaber AJ, Armstrong EG. An online spaced-education game to teach and assess medical students: a multi-institutional prospective trial.  Acad Med. 2012; Oct;87(10):1443-1449.

This trial involved 731 students at 3 US medical schools and showed that the spaced education methodology combined with game mechanics is an effective and well-accepted means of teaching core content and is a reliable and valid method to assess student knowledge.

Kerfoot BP, Baker H.  An online spaced-education game to teach and assess residents: a multi-institutional prospective trial. J Am Coll Surg. 2012 Mar;214(3):367-73.

Involving 931 urology residents from the US and Canada, this trial demonstrated that the spaced education methodology combined with game mechanics is a reliable and valid means to assess residents’ knowledge and is a well-accepted method by which residents can master core content.

Shaw T, Long A, Chopra S, Kerfoot BP. Impact on clinical behavior of face-to-face continuing medical education blended with online spaced education: A randomized controlled trial. J Contin Educ Health Prof. 2011 Mar;31(2):103-8.

This randomized trial among 246 clinicians provides evidence that spaced education following a live CME course can significantly increase the impact of a face-to-face course on providers’ self-reported global clinical behaviors. This study won the 2012 Award for Excellence in Research, from the Journal of Continuing Education in Health Professions, recognizing “the best research article published in JCEHP” in 2012.

Kerfoot BP, Shaffer K, McMahon GT, Baker H, Kirdar J, Kanter S, Corbett EC Jr, Berkow R, Krupat E, Armstrong EG. Online “spaced education progress-testing” of students to confront two upcoming challenges to medical schools.  Acad Med. 2011 Mar;86(3):300-6.

This 34-week randomized trial involving 1067 students at 4 US medical schools demonstrated that the spaced education methodology can identify poorly performing students and improve their longer-term knowledge retention by 170%.

Kerfoot BP, Lawler EV, Sokolovskaya G, Gagnon D, Conlin PR. Durable improvements in prostate cancer screening from online spaced education a randomized controlled trial. Am J Prev Med 2010;39:472-8.

This 108-week randomized trial involving 95 clinicians in 8 northeastern hospitals showed that the spaced education methodology durably improves the prostate cancer screening behaviors of clinicians by 40%. This is the first study demonstrating that spaced education can durably impact the behaviors of participants.

Kerfoot BP, Fu Y, Baker H, Connelly D, Ritchey ML, Genega EM. Online spaced education generates transfer and improves long-term retention of diagnostic skills: a randomized, controlled trial. J Am Coll Surg. 2010 Sep;211(3):331-337.

In this 45-week randomized trial involving 724 urology residents across the United States demonstrated that the spaced education methodology generates transfer of histopathology diagnostic skills and substantially improves their long-term retention.

Long, A, Kerfoot BP, Chopra S, Shaw T. Online spaced education to supplement live courses. Med Educ 2010; 44, 519-20.

This pilot study demonstrates that online spaced education is an effective and well-accepted supplement to a live CME course.

Kerfoot BP. Adaptive spaced education improves learning efficiency: a randomized, controlled trial.  J Urol 2010; 183(2), 678-681.

This randomized trial provides evidence that the spaced education methodology with adaptive game mechanics can increase learning efficiency by more than 35%.

Kerfoot BP, Kearney MC, Connelly D, Ritchey ML. Interactive spaced education to assess and improve knowledge of clinical practice guidelines: a randomized controlled trial. Ann Surg, 2009 May; 249(5):744-9.

This randomized trial among 480 clinicians demonstrated for the first time that the spaced education methodology is an effective and well-accepted form of graduate- and continuing-medical education and is a promising new methodology to improve knowledge of clinical practice guidelines.

Kerfoot BP. Learning benefits of online spaced education persist for two years. J Urol 2009. Jun;181(6): 2671-3.

This study analyzed whether the learning gains among the residents in our 2007 spaced education trial persisted after two years.  They did!  These results are particularly striking since these residents’ on-the-job training over these 2 years would have worked to erode any knowledge differences between the cohorts that were attributable to the spaced education methodology. Given this, the ability to demonstrate a  knowledge difference between cohorts 2 years after the spaced education intervention is remarkable.

Matzie KA, Kerfoot BP, Hafler JP, Breen EM. Spaced education improves the feedback that surgical residents give to medical students: a randomized trial. Amer J Surg 2009: 197(2), 252-257.

This randomized trial employed the spaced education methodology to improve the feedback that surgical residents give to medical students.  The medical students were blinded to which residents received the spaced education.  Even so, the students (the blinded, down-stream receivers of the feedback) reported that the feedback given by residents who received the spaced education was significantly more frequent and of higher quality.

Kerfoot BP, Armstrong EG, O’Sullivan PN. Interactive spaced education to teach the physical examination: a randomized controlled trial. J Gen Intern Med. 2008; 23(7):973-8.

This 18-week randomized trial among medical students at Harvard demonstrated that the spaced education methodology can generate significant improvements in knowledge of the physical examination and is very well-accepted by students.

Kerfoot BP, Brotschi E. Online spaced education to teach urology to medical students: a multi-institutional randomized trial. Amer J Surg 2009;197(1):89-95.

This randomized trial among 115 medical students at Harvard provides evidence that spaced education delivered prospectively can generate significant, topic-specific learning.

Kerfoot BP, Armstrong EG, O’Sullivan PN. Impact of item clustering on interactive spaced education. Medical Education 2008; 42: 1115–1116.

This pilot randomized trial among 97 medical students at Harvard showed that moderate clustering of content within the spaced education methodology does not have a large effect on learning and retention. A substantially larger trial (128–788 students for 0.8 power) is needed to investigate if clustering has a small-to-moderate impact on learning and retention.

Kerfoot BP. Interactive spaced education versus web-based modules for teaching urology to medical students: a randomized controlled trial.
J Urol 2008; 179, 2351-2357.

This randomized trial demonstrated that, within the compact time frame of a clinical clerkship, spaced education is equivalent to web based teaching in short-term learning gains and in acceptability by medical students.  In addition, the study found that ISE is able to generate substantial improvements in knowledge from one cycle to the next.

Kerfoot BP, Baker HE, Koch MO, Connelly D, Joseph DB, Ritchey ML. Randomized, controlled trial of spaced education to urology residents in the United States and Canada. J Urol. 2007; 177:1481-1487.

This randomized trial among 537 residents across the United States and Canada demonstrated for the first time that the spaced education methodology improves the acquisition and retention of clinical knowledge.

Kerfoot BP, DeWolf WC, Masser BA, Church PA, Federman DD. Spaced education improves the retention of clinical knowledge by medical students: randomized controlled trial. Med Educ, 2007; 41:23-31.

This is the first randomized of the spaced education methodology.  The study demonstrates that spaced education consisting of clinical scenarios with questions and explanations distributed weekly can significantly improve students’ retention of medical knowledge.

Top