"Does it work?"

What is Efficacy?

Efficacy is the research question: does this work? Not "do teachers like it", not "do students seem engaged" — does the learning we claim to produce actually happen, measured in a way that would survive scrutiny from someone with no stake in the answer.

Efficacy evaluates whether a tool actually delivers the learning outcome it promises, under rigorous research conditions. — ICEI 5E Standards & Definitions

ICEI splits it into two parts. The research basis is the argument: a theory of change or logic model that says what the product does, what that should cause, and why anyone should believe the link. The evidence is what has actually been observed, and it sits on a ladder — studies with no comparison group at the bottom, studies with a comparison group in the middle, randomised controlled trials at the top.

The two are not interchangeable. A beautiful theory with no evidence is Bronze. Evidence with no theory behind it is hard to interpret. Efficacy wants both, in that order.

Why it matters

Schools are asked to adopt digital tools constantly, and the case is almost always made with satisfaction data — how many teachers signed up, how many said they would recommend it. That tells you a product is pleasant. It does not tell you a student learned anything.

Efficacy is the dimension that closes that gap, and it is deliberately difficult. The ladder is public, the rungs are specific, and you cannot talk your way up them: a comparison-group study either exists or it does not. For a school or a ministry deciding between platforms, it is the one dimension where the answer cannot be spun.

It matters to us for a second reason. Class2Class makes a strong claim — that a collaborative project with a classroom in another country develops competencies that a domestic project would not. That is exactly the kind of claim that feels obviously true and is rarely tested. If it is true, we should be able to show it. If it is not, we should want to know.

How we work on Efficacy

We work on Efficacy in three places: in what the model commits to, in what the platform captures, and in who we ask to check us.

Saying in advance what should change

The educational model names eight competencies and four levels of a Changemaker Journey. That is the outcome set — the thing a project is supposed to move. Naming it in advance is what makes the claim testable rather than retrospective.

Making the evidence legible

• Every learning outcome is tagged to a Bloom’s Taxonomy level, so an outcome carries a cognitive level and not just a sentence.
• A group’s age is captured as the first step of project setup, so every later suggestion arrives calibrated to developmental stage rather than corrected for it afterwards.
• The evaluation architecture collects a teacher checklist, a group reflection at project closure, and optional evidence tiers — and turns them into a group-level narrative the teacher confirms.

Asking someone else

In 2025 we commissioned the International Centre for EdTech Impact — an independent research organisation in Egersund, Norway — to examine the model rather than writing our own review of ourselves. Their report was delivered in January 2026.

What we are building next

• A link, in the data model, between each learning outcome and the competencies it develops — so classroom evidence can roll up into something countable.
• An outcome measure per competency: unit, scale, scoring rule, and who scores it. Without this there is no dependent variable for any future study.
• A scope for a comparison-group study, as a new and separate commission.

Related reading