Duolingo: What Should a Learning Product Optimise After the Streak?

Duolingo once tested a change that made a streak easier to keep. Learners could extend their streak with a single lesson, and their daily goal was tracked separately. In Duolingo's account of the experiment, Day 14 retention rose by a relative 3.3%.

The change increased retention even as fewer learners reached their daily goals. Duolingo also reports that aggregate learning time increased. Return, daily-goal completion and total practice therefore gave different signals, so a team that watches only one of them can misread the others. This was a historical test, not a current effect size, and it did not establish what happened to language ability. That needs a separate measure.

This note asks what a learning product should optimise once the habit exists. Our argument is that the streak should stay as a low-friction reason to return, and that changes to the learning experience should be judged by what a learner can retrieve and use later, with less task-specific help. Daily activity is an input to that outcome, not a substitute for it. That is our position, not Duolingo's.

We use "learning outcome" narrowly. It means a learner can use taught language after a delay, on a task that resembles the practised one without repeating it. That is not fluency, and it is not a claim about what every learner wants.

What we looked at, and what we did not

Duolingo is a freemium learning company whose flagship app combines short lessons, practice, adaptive instruction and game mechanics. Its Q2 2026 shareholder letter reports 58.7 million average daily active users during Q2 and 12.7 million paid subscribers at quarter-end. Those are different populations measured over different periods, so dividing one by the other does not give a conversion rate. The same letter says management is prioritising retention and learning outcomes this year, with monetisation second. That is a statement of intent, not a measured result.

We read Duolingo's shareholder letters, product and research pages and engineering blog, plus the original research papers, checked on 30 October 2026. That was Q2 2026, the latest quarter available at that date. We have not used a learner account, seen internal data, interviewed anyone at the company or run an experiment. Nothing here establishes a learning problem or a retention problem.

The scope is adult language learning. Maths, music, chess, children's learning and the Duolingo English Test need different outcome definitions. "After the streak" does not mean switching the habit off on day 30. It means looking beyond habit counts when judging what to build.

What the streak does, and what it cannot show

The streak has a clear job. It gives a learner a small daily reason to return, and the experiment above suggests Duolingo has tuned how large that reason needs to be. In June 2026 it also let eligible learners restore their longest streak by completing three lessons. The Q2 letter reports that about 15.4 million people did so, including nearly eight million with no active streak at the start. That shows reactivation. It does not tell us how much of the language those learners still had.

Our reading is that the streak protects access to a learner's future attention. A busy day can reasonably call for a short maintenance session, and demanding a hard assessment every day could cost more habit than it buys. But showing up cannot settle whether the material was understood, remembered or usable somewhere new. Both things can hold: a streak may help learning by keeping people present, and a streak alone cannot measure what they learned.

Signals that answer different questions

The table is our interpretation of what each measure supports. A proxy can be useful without being a complete outcome, and every assessment has measurement error and limited coverage.

Duolingo already moved beyond raw activity. Its Time Spent Learning Well explanation describes replacing session counts with weighted learning time. Minutes on the learning path count in full and other learning minutes at half weight, and XP and quests were adjusted to reward progression. That is an existing response to easy-session grinding. It does not show that every easy session is wasteful, and as a public explanation it may not match the metric exactly as used inside the company today. Review can also be the right activity. Someone consolidating fragile knowledge should not be pushed into new material just to lift an activity proxy, and slower completion is not automatically deeper learning, because it can reflect difficulty, distraction or friction. These are reasons to check activity signals against later performance.

Progress is not certification. The Duolingo Score rises as learners finish lessons and move through a course, and it maps course content to CEFR levels. It uses a 0 to 160 range like the Duolingo English Test, but the app score and the separate test are different things. Covering B2-level material does not show that a learner performs at B2 across skills. The Council of Europe describes CEFR levels as descriptors of increasing mastery within types of competence, written without reference to any one language. We read a level as a way of specifying ability, not something earned by finishing a path. A claim should name the skill, the task, the conditions and the assessment behind it.

What Duolingo already provides

A useful proposal cannot treat personalised review, scenario practice, speaking or end-of-unit checks as missing. They already exist.

Features vary by course, platform and plan, and older posts can describe earlier stages of the product, so check the source and date. The proposal later in this note depends on one thing we could not check: whether the exercise we pick already asks for an unsupported attempt before it shows a model.

What the learning evidence supports

The research is neither "only engagement numbers" nor proof of fluency. Different studies support different outcomes, and company involvement is a reason to read the method closely, not to discard the result.

The Spanish paper also records that some participants wanted more speaking support, even though speaking was among the areas with the largest measured gains. That is a reason to measure experience and performance separately. It does not mean the participants were wrong or that today's speaking features fall short, because the study predates important product changes.

A disciplined reading is that learners can improve using Duolingo, including on tasks beyond recognition. The public evidence we reviewed does not show how delayed transfer is spread across all active learners, every course, plan or type of practice. It also says nothing about the sequencing change proposed below.

Support now, independence later

Models, hints, word banks and repetition help a learner begin and succeed. Removing them too early can produce frustration instead of learning. Leaving task-specific support in place forever makes success harder to interpret, because we cannot tell whether the learner produced the language or followed an answer on the screen.

That does not make maximum difficulty the goal. A familiar phrase produced independently can be worth more than a confusing new task, and repetition can build automaticity. What matters is the match between support, readiness and the communication the learner is aiming at.

Original research gives a reason to separate immediate success from delayed performance. In Roediger and Karpicke's 2006 experiments, rereading beat repeated testing on a test five minutes later, but tested students remembered much more after two days and after a week. Butler (2010) found better retention and transfer to new questions a week later after repeated testing than after repeated study. Butler and colleagues (2017) found that retrieving and applying knowledge across different examples improved transfer to new examples two days on. All three used prose or lecture material, not language lessons, so they support a mechanism worth testing and not an effect size for Duolingo.

Our decision rule, which is a proposal: reduce task-specific support when there is enough prior instruction for a first attempt to be reasonable, keep help accessible, and keep the change only if later communication improves enough to justify any added burden or loss of participation. "Later" matters because the training task is an unfair finish line. "Unfamiliar" matters because repeating the same sentence may reflect rehearsal. Neither word means an untaught situation or an external exam every day.

What a learning product can promise

Management ties useful learning to growth. The Q1 letter says teaching better can drive engagement, retention, growth and brand credibility. That is a business hypothesis, and the public material does not show how much a specific gain in delayed ability causes renewal or referral.

The table is our go-to-market interpretation. A learner preparing to talk with relatives, travel or enjoy films has different success criteria, and speaking is an example here, not a definition of learning value. A strong claim names the population and the outcome. "Fluent because you have a long streak" would not qualify.

One improvement is to demonstrate a bounded capability under transparent conditions, such as handling a previously unseen variation of a taught task. We would not invent a new public certificate or readiness badge. Duolingo already invests in the Score and in proficiency positioning, and another signal could add confusion before it has been validated.

A subscriber who leaves after reaching a goal may still have received value, and lower use can reflect frustration, seasonality or another product. Neither continued use nor cancellation settles whether learning succeeded, so the reasons matter more than the exit count.

A test with one change

Hypothesis (ours). For learners who have already met the required language, attempting a response before seeing a model answer improves delayed performance on new prompts enough to justify any extra effort, without an unacceptable loss of participation.

Segment. Consenting adult learners in one language pair, for example English speakers studying Spanish, at a stated point in the course. They have already studied the target vocabulary and grammar, they return regularly, and they want to practise speaking. Eligibility is set from baseline data before assignment, because we found no evidence for a universal streak-length cut-off.

Audit first. Pick one real model-supported review slot, such as repeating a model sentence aloud, and confirm in a walkthrough that it does not already ask for an unsupported attempt. If it does, the test adds nothing there and should not run. We do not know Duolingo's internal sequencing rules. The test also needs a small prototype, because we found no public setting that changes when the model appears, and we do not claim the idea is missing everywhere in the product.

Control. Show or play the model, then ask for a response.

Treatment. Ask for a first attempt, then show or play the same model. A "show me an example" route stays one tap away, and using it is recorded and not penalised.

Both versions use the same situational cue, taught language, response task, corrective feedback and completion credit. We would change nothing in existing leaderboard or streak mechanics, and add no upsell, sharing prompt or proficiency badge. A harder first attempt must not cost extra currency, Energy or streak progress, or a drop in participation could reflect a different penalty and not the instructional challenge. Those controls are in the methodology appendix.

Main outcome. Performance after a delay set before assignment (a week is illustrative) on unseen, matched prompts that need the same taught functions, for example making a request, understanding a changed detail and answering a short follow-up with known language. A written rubric scores communicative success, comprehension and intelligibility, allows more than one valid answer, and is applied by assessors who do not know the group where feasible. This is a bounded simulation of transfer. It is not proof of conversation in real settings.

The trade-off to watch. A harder first attempt may build durable skill, and it may also cost time, add frustration or make people stop. So alongside the delayed outcome we track return, continued study, help use, actual time and signs of distress against limits agreed in advance. Confidence, perceived usefulness and paid outcomes are recorded separately. No pricing or upgrade message changes during the test.

Randomise at learner level, analyse learners as assigned, and follow up people who stop using the app, because looking only at those who stay would flatter the treatment. Begin with qualitative work on difficulty, accessibility and scoring, then size a controlled test from baseline variation. The full plan is in the appendix.

Reading the result. We would recommend expanding the change only if delayed performance improves by a practically worthwhile amount, participation stays within the limits agreed, and the result survives reasonable assumptions about missing follow-up.

• Better delayed performance, but only among a small group who stay while many leave: look at the whole cohort before recommending rollout.
• Gains only on the practised sentence: the transfer idea is unsupported.
• Better results because the treatment simply adds study time: say so plainly, because that is a different finding.
• No detectable difference: the test found no benefit under its conditions. With small samples the uncertainty is large, so this does not show the two designs are equivalent.
• The chosen slot already starts with an attempt: there is no contrast to test.

A measurement ladder after the streak

We would not replace the streak with one new score. Our recommendation is a ladder of evidence, with fast activity measures for daily iteration, periodic validation of those measures against durable skills, and a check that those skills serve what the learner wanted to do.

1. Habit: are people able and willing to return?
2. Practice quality: are they meeting and revisiting appropriate material, with support that fits their readiness?
3. Durable performance: can they use taught functions later, on unfamiliar but appropriate tasks?
4. Goal attainment: does that ability help with the activity they actually wanted to do?

This is our synthesis and is compatible with Duolingo's own efficacy framework. A daily experiment does not need a full proficiency exam for every user, because sampled assessments can calibrate faster proxies, provided the sample and the missing outcomes support the claim being made. Two cautions apply. "Learning per minute" can be improved by dropping learners who need more help or by choosing easy-to-measure skills, so report absolute gains, the spread of outcomes, participation and effort together. And assessment success should not become a new streak requirement, because that turns measurement into punishment.

The case against changing anything

• The current system may already make the right trade-off. Duolingo personalises and tests heavily, and this sequence may already be in use.
• Models may be necessary at this level. An unsupported attempt can be unproductive when the language has not yet been learnt.
• Repetition is not automatically gaming. Pronunciation, automaticity and confidence can all benefit from familiar material.
• Learners may want something other than spontaneous speech. Reading, enjoyment and staying in touch with a language are legitimate goals.
• Tests change behaviour and cost time. An intrusive assessment programme could reduce the learning it tries to measure.
• A simulation is still a simulation. A short task does not establish sustained conversation or performance under stress.
• Engagement and learning can improve together. Nothing here shows a fixed conflict, or that Duolingo avoids teaching well.

So the current supported practice is the right choice when a learner is still meeting the language, when an unsupported attempt would mostly produce guessing, or when the goal is to keep a fragile habit alive. The proposal applies to learners who have already met the language and are ready to be asked for it.

What we do not know:

• How delayed gains are distributed among all active learners.
• How Duolingo's activity proxies are validated against later performance.
• Which exercises already fade support, and when.
• Which features each individual account can use today.
• What the proposed change would do to learning, retention, referrals or revenue.

What would weaken our thesis: if delayed assessments already govern these design decisions, this note describes good practice and not a product gap. If model timing is not the binding constraint, a different intervention is needed. And if learners' chosen goals do not need productive speaking, the outcome should change before the practice does.

Next step

Product Garage works through live product and go-to-market decisions with experienced operators. This note takes one of those decisions in public, while the thinking is still provisional.

Subscribe to Garage Dispatch to get the next working note.

FAQ

Does Duolingo measure learning, or only streaks and activity? It measures both, and it publishes more than activity. Its Time Spent Learning Well metric weights minutes on the learning path above other minutes, and its efficacy framework separates engagement, mastery, application and proficiency. The public material does not show how far delayed performance feeds product decisions day to day.

Does keeping a streak mean you are learning? A streak shows that someone returned. Duolingo's streak experiment reported better Day 14 retention, which is about coming back and not about skill gained. Whether the material is remembered and usable later needs a separate measure.

Is a Duolingo Score the same as a CEFR level or a Duolingo English Test result? No. The Score rises with course progress and maps course content to CEFR levels, and it uses the same 0 to 160 range as the Duolingo English Test. They are separate things, and completing B2-level material does not show that a learner performs at B2 across skills.

Does Duolingo teach speaking? It has several speaking features, including spoken responses, oral flashcards, Speaking Adventures and Video Call. Company-reported studies show gains on specific speaking tasks, some narrower than spontaneous conversation. The public evidence does not show how those gains are spread across all learners or how well they carry to unfamiliar situations.

What does "learning outcome" mean in this note? A learner can use taught language after a delay on a task that resembles the practised one without repeating it. That is our working definition for this argument, not a metric Duolingo publishes.

Methodology appendix

This appendix supports the working note above. It is a proposal, not a report of a test that has been run.

Conditions held equal

• Learners: consenting adults in one language pair, at a stated point in the course, who have already studied the target vocabulary and grammar. Eligibility is set from baseline data before assignment. There is no evidence here for a universal streak-length threshold.
• The slot: one real model-supported review slot, chosen after a live walkthrough that confirms it does not already ask for an unsupported attempt before showing a model. If it does, the test is not run there.
• Content: the same situational cue, taught language, response task and corrective feedback in both groups.
• Exposure: the same intended model exposure, task count, planned time budget, reminders and access. Actual time and help use are outcomes to record, not quantities assumed to be equal.
• Consequences: the same completion credit, and no extra currency, Energy, streak or mistake-related penalty for a harder first attempt in this activity. Record the surrounding account configuration.
• Nothing bundled: no changes to existing leaderboard or streak mechanics, and no added upsell, pricing change, sharing prompt or proficiency badge in either group.
• The one difference: the model appears before the first attempt (control) or after it (treatment). In the treatment, a "show me an example" route stays one tap away, and its use is recorded and not penalised.

Human curriculum review should confirm that a reasonable first answer is possible from previously taught material. The treatment needs a small exercise prototype or product experiment. No public setting was found that changes when the model appears, and the prototype is not described as a capability missing from Duolingo.

Outcome and scoring

• Primary outcome: performance after a delay set before assignment, on unseen, matched communication prompts that need the same taught language functions. A week is illustrative. Choose the interval from the learning goal and feasibility.
• Prompt pool: a separate assessment pool, not the rehearsed sentences or the training character's predictable turns. For example, make a request, understand a changed detail, and respond to a short follow-up using known language.
• Rubric: written in advance. It covers communicative success, relevant comprehension and intelligibility, and allows more than one valid answer. Asking for clarification can be successful communication and is not automatically an error. A native-like accent is not the goal.
• Assessors: qualified human raters, who do not know the assigned group where feasible. If automated scoring is used, check it against human ratings on an appropriate sample, including variation across accents and recording conditions.
• Limit of the inference: this is a bounded simulated transfer task. It does not show sustained conversation, performance under stress or general fluency.

Measures

Design notes

• Randomise at learner level for the defined activity. Avoid assigning different policies to the same person in ways that contaminate later assessment.
• Record baseline ability, not only streak length. Balance or stratify on account plan, platform, baseline performance and prior exposure to the language.
• Qualitative first. Observe a few learners to check comprehension, suitable difficulty, accessibility and reliable scoring. Use baseline variation to size a later controlled test. Do not invent a sample size or an uplift target.
• Predeclare the delay, the rubric, the acceptable participation limits and the smallest gain worth acting on.
• Analyse as assigned. Keep learners who request help, skip the activity or stop using the app in their original group, and report exposure separately.
• Missing outcomes. A missing proficiency assessment is neither a zero nor evidence of success. State the missing-data assumptions and run sensitivity analyses. A completers-only result cannot support a claim about the whole population.
• Do not adjust away study time in the main analysis. Extra effort or reduced practice after assignment can be part of the effect. A separate time-matched mechanism study answers a different question and is not the primary policy test.
• Success rule. Recommend expansion only if delayed performance improves by a practically worthwhile amount, participation costs stay within the agreed limits, and the inference survives reasonable missing-data assumptions. Otherwise report harm, no detectable benefit or an inconclusive result. A small sample and a null result do not show the two designs are equivalent.
• Read the pattern. If the treatment simply adds study time, describe it that way. If gains appear only on the trained sentence, the transfer hypothesis is unsupported. If performance improves only among a small group that stays while many leave, examine the whole-cohort consequence before recommending rollout.

Reward and penalty controls

Learners have finite time, confidence and access to practice, and paid convenience, AI cost and free-user restrictions can influence which activities people choose. The implemented experience is therefore part of the test. Duolingo's Energy explanation describes a rollout to a small group of users that was being tested and likely to change, with Energy spent as lessons are completed and replenished by consecutive correct answers and other routes. Do not assume every free learner now follows the same rules, or that older Hearts descriptions are obsolete. Equalise these consequences within the experimental activity so that a fall in participation cannot be traced to a different currency penalty.

Measurement traps

• Streak or return rate shows habit continuity. It does not show time spent, skill gained or practice on every day the streak represents.
• XP is rewarded activity within an incentive system that Duolingo has changed over time, so it is not a stable unit of proficiency.
• Time Spent Learning Well favours particular practice. It is a proxy to validate against later performance, not a learning result.
• Score or course position shows progress through material. It is not an independent assessment of each skill at that level.
• Unit mastery checks assess the unit's concepts. They do not establish long-term retention or transfer.
• Confidence records experience and preparedness. It does not record performance.
• "Learning per minute" can be improved by dropping learners who need more help or by choosing easy-to-measure skills. Report absolute gains, the distribution of outcomes, participation and effort together.
• Assessment success as a streak requirement confuses measurement with punishment.
• Continued use and cancellation are each ambiguous. A learner may leave after reaching a goal, or leave out of frustration or seasonality, or move to another product.