Study Protocol

Between-subjects, assigned at consent from a cryptographic random source. The browser receives an opaque group label, so the condition name never reaches the interface that could leak the manipulation.

The fixed schedule the control condition serves: task 1 at level 3, task 2 at level 2, task 3 at level 1.

Three levels of support, most to least. The same three are available to both conditions; what differs is how the level for each task is chosen.

• Level 3

  Full worked example

  complete Python solution shown with annotations

• Level 2

  Partial completion

  starter code with blanks to fill

• Level 1

  Minimal support

  problem statement only, independent practice

The learning activity, in order. The framing is published; the prompts, worked examples, starter code, hints and test cases are part of the embargoed material and are given here only as counts.

• Task 1

  Iterating Through a ListPlain iteration

  You're browsing books in the library catalog. Let's display each book's price.

  Withheld: 4 worked-example annotations, 3 hints, 1 test case.

• Task 2

  Finding Books Under BudgetFiltering on a condition

  You have $10 to spend. Let's find which books you can afford.

  Withheld: 4 worked-example annotations, 2 hints, 1 test case.

• Task 3

  Calculating Total CostAccumulation

  You're checking out with several books. Let's calculate your total.

  Withheld: 3 worked-example annotations, 2 hints, 1 test case.

Item counts are published. The items themselves are not.

• Pretest

  3 items

  Items embargoed

  Multiple choice, taken before the learning activity as the prior-knowledge measure.

• Posttest, multiple choice

  10 items

  Items embargoed

  Syntax, prediction, error identification and transfer items.

• Posttest, coding challenge

  1 item

  Items embargoed

  Free-response Python. The server stores the submission without grading it, because the item carries no answer key, and it is scored afterwards against 5 rubric criteria: syntax, indentation, variable, executes, correct_output.

The subscale structure and the response range are published because the source instrument is open access, so nothing here is revealed that the paper does not already give away. The item wording used in this study stays withheld with the rest of the instrument.

Three agents with separate jobs, each on a different model family so no single model's habits decide how much help a learner gets. Decisions are computed from a lookback window of 2 tasks, bounded to levels 1 through 3, starting every learner at level 3.

The antagonist argues from a versioned learning-theory knowledge base, currently version 1.2.0, last updated 2026-08-24. The decision record does not carry that version, so it is stated here instead, and the build fails if the knowledge base is updated without this protocol being revised alongside it.

One decision record, field by field

The fields are the columns the study writes for every adaptive decision. The values are an illustration of the case the antagonist exists for, where the worker proposes fading two levels at once and is overruled. No participant has been run, so this is not a record of anyone.

Five directional predictions, each with the test it is judged by and the threshold it is judged at. They were fixed before any participant was recruited, which is the only claim about them worth making before there is data.

• H1

  extraneous_load_mean

  alpha = .0167

  The AI-adaptive condition will produce lower extraneous cognitive load than the static faded condition.

  Test: One-sided Welch independent-samples t-test.

  The mechanism the whole study rests on. Reported with the mean difference, a 95% confidence interval and Hedges' g.

• H2

  germane_load_mean

  alpha = .0167

  The AI-adaptive condition will produce germane cognitive load that is not meaningfully lower than the static faded condition.

  Test: One-sided Welch independent-samples t-test. Secondary: TOST equivalence test, bounds d = plus or minus 0.30.

  Guards against the adaptive system helping so much that it suppresses the effort schema construction needs. If the superiority test is not significant, the equivalence test decides whether the two conditions are practically equivalent instead of leaving a bare failure to reject.

• H3

  intrinsic_load_mean

  alpha = .0167

  There will be no practically meaningful difference in intrinsic cognitive load between conditions.

  Test: Two-sided Welch independent-samples t-test. Secondary: TOST equivalence test, bounds d = plus or minus 0.30.

  A manipulation check. The material is identical in both conditions, so equivalent intrinsic load is the evidence that the manipulation changed the quality of support rather than how hard the task felt. Reported with a 90% confidence interval, which is what TOST requires.

• H4

  posttest_mc_score

  alpha = .05

  The AI-adaptive condition will produce higher posttest performance than the static faded condition, after adjusting for pretest performance.

  Test: ANCOVA with heteroscedasticity-consistent (HC3) standard errors, regressing posttest score on condition and pretest score. Secondary: Welch t-test on the raw posttest score, plus descriptive gain scores.

  The learning outcome. Tested at the uncorrected threshold because it sits outside the cognitive load family.

• H5

  final_scaffold_level by problem number, adaptive condition only

  alpha = .05

  Within the AI-adaptive condition, scaffold level will fall as task position advances, which is what contingent fading looks like in the decision trace.

  Test: Cumulative-link mixed model with participant-level random intercepts, if sample size permits. Secondary: Pre-specified fallback if the model is unstable: Spearman association plus a descriptive transition analysis of fade, maintain and increase after correct and incorrect responses.

  A mechanism check on the adaptive system's own behavior, carrying lower inferential weight than H1 to H4 because each participant contributes only three decisions.

An a priori power analysis, not a number chosen for convenience. Both sample sizes below are real: one is the floor, the other is what the multiplicity correction costs.

The populations, the exclusions and the outcome variables, all fixed in advance. Exclusions are pre-specified rules recorded by the running system, not judgements made once the data is in.

Complete-case analysis for each endpoint, with no imputation. Missingness is reported by condition and by phase, and technical terminations in the adaptive condition are reported separately rather than folded into attrition.

Reporting follows CONSORT: a participant flow diagram, a baseline table by condition, effect sizes with confidence intervals for every primary test, the count and reason for each exclusion, and primary against sensitivity results side by side.

The confirmatory analysis is run in statistical software against the exported dataset. The in-site dashboard computes Welch's t-test, Cohen's d, TOST and Spearman for monitoring, and its ANCOVA is an ordinary-least-squares fit without the HC3 correction, so it is not the confirmatory model.

Each reference is here because a specific decision in the experiment exists because of it. Anything that could not be tied to a decision was left off.

• Sweller (1988)

  Theory

  The reason cognitive load is the outcome rather than a covariate. Working memory is the binding constraint on schema construction, so an instructional change is measured by what it does to the load it imposes, not only by a score at the end.

  Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.

• Sweller, Van Merriënboer, & Paas (1998)

  Theory

  The three-way split of load into intrinsic, extraneous and germane, which is the structure of the measurement and of H1 to H3: extraneous is what adaptive support should reduce, germane is what it must not suppress, and intrinsic is the manipulation check.

  Sweller, J., Van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296.

• Kalyuga, Ayres, Chandler, & Sweller (2003)

  Theory

  Why support is withdrawn on evidence of competence rather than on a clock. Guidance that helps a novice becomes redundant once a schema exists, and processing it then costs working memory. That is the failure the adaptive condition is built to avoid and the fixed schedule is built to expose.

  Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23–31. https://doi.org/10.1207/S15326985EP3801_4

• Wood, Bruner, & Ross (1976)

  Theory

  The definition of scaffolding the whole manipulation inherits: control of the parts of a task that exceed a learner's current competence, withdrawn as competence grows. The three support levels are that idea made discrete enough to assign and to log.

  Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17, 89–100.

• Vygotsky (1978)

  Theory

  The zone of proximal development, which is the band the scaffold levels are meant to keep a learner inside and one of the three checks the antagonist agent argues from when it objects to a proposed level.

  Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.

• Sweller & Cooper (1985)

  Theory

  The worked-example effect, which is why the comparison condition is faded worked examples rather than unguided practice. The control has to be a treatment that works, or a difference in favor of the adaptive condition says nothing.

  Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem-solving in learning algebra. Cognition and Instruction, 2(1), 59–89.

• Renkl (1997)

  Theory

  The observation that the right moment to fade differs between learners, because self-explanation quality does. That individual variation is the gap a fixed 3-2-1 schedule cannot close and the adaptive condition is testing.

  Renkl, A. (1997). Learning from worked examples: A study on individual differences. Cognitive Science, 21(1), 1–29.

• Shin et al. (2025)

  Theory

  Fading as the bridge between studying a solution and generating one, in a programming context. It is the reason the levels run complete example, partial code, prompt alone rather than switching support off in one step.

  Shin, Y., Jung, J., Choi, S., & Jung, B. (2025). The influence of scaffolding for computational thinking on cognitive load and problem-solving skills in collaborative programming. Education and Information Technologies, 30, 583–606. https://doi.org/10.1007/s10639-024-13104-0

• Faber et al. (2024)

  Theory

  Contingent, performance-based fading as the thing being tested, and the warning that comes with it: a fixed schedule can match a learner's needs by coincidence often enough to erode the difference. The fixed schedule here is strictly non-responsive so that coincidence is at least measurable.

  Faber, T. J. E., Dankbaar, M. E. W., van den Broek, W. W., Bruinink, L. J., Hogeveen, M., & van Merriënboer, J. J. G. (2024). Effects of adaptive scaffolding on performance, cognitive load, and engagement in game-based learning: A randomized controlled trial. BMC Medical Education, 24(943). https://doi.org/10.1186/s12909-024-05698-3

• Bjork & Bjork (2011)

  Theory

  The constraint on the whole adaptive design: performance during learning is a poor proxy for learning, so a system that minimizes difficulty is not the same as a system that helps. The antagonist agent exists to argue against over-support for this reason.

  Bjork, E. L., & Bjork, R. A. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In M. A. Gernsbacher, R. W. Pew, L. M. Hough, & J. R. Pomerantz (Eds.), Psychology and the real world: Essays illustrating fundamental contributions to society (pp. 56–64). Worth Publishers.

• Bjork & Bjork (1992)

  Theory

  The account of retrieval and storage strength behind desirable difficulties, and the reason H2 is framed as germane load not being suppressed rather than as load being reduced across the board.

  Bjork, R. A., & Bjork, E. L. (1992). A new theory of disuse and an old theory of stimulus fluctuation. In A. F. Healy, S. M. Kosslyn, & R. M. Shiffrin (Eds.), From learning processes to cognitive processes: Essays in honor of William K. Estes (Vol. 2, pp. 35–67). Erlbaum.

• Zimmerman (2000)

  Theory

  The cyclical model of self-regulated learning that the adaptive condition externally enacts: the system monitors and adjusts on the learner's behalf, then hands that responsibility back as support fades.

  Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13–39). Academic Press.

• Zimmerman (2008)

  Theory

  The methodological case for measuring regulation through process traces rather than self-report alone, which is why every scaffold decision, keystroke count and pause is recorded alongside the survey.

  Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45(1), 166–183. https://doi.org/10.3102/0002831207312909

• Klepsch, Schmitz, & Seufert (2017)

  Method

  The cognitive load instrument itself, and the confirmatory factor analysis that justifies keeping three subscales rather than two. The subscale structure the study administers comes straight from it.

  Klepsch, M., Schmitz, F., & Seufert, T. (2017). Development and validation of two instruments measuring intrinsic, extraneous, and germane cognitive load. Frontiers in Psychology, 8, Article 1997. https://doi.org/10.3389/fpsyg.2017.01997

• Lakens, Scheel, & Isager (2018)

  Method

  The equivalence procedure behind H2 and H3. A manipulation check whose desired outcome is a null result cannot rest on failing to reject, so both carry a two one-sided test against explicit bounds.

  Lakens, D., Scheel, A. M., & Isager, P. M. (2018). Equivalence testing for psychological research: A tutorial. Advances in Methods and Practices in Psychological Science, 1(2), 259–269. https://doi.org/10.1177/2515245918770963

• Klasen & Sopka (2020)

  Method

  The reason the equivalence bounds are set where they are, following recommendations for educational intervention research rather than a threshold chosen after seeing the data.

  Klasen, M., & Sopka, S. (2020). Demonstrating equivalence and non-inferiority of medical education concepts. Medical Education, 55(4), 455–461. https://doi.org/10.1111/medu.14420

• Long & Ervin (2000)

  Method

  The HC3 standard errors used in the H4 model, chosen because the posttest score is a bounded count and there is no reason to assume equal residual variance across conditions.

  Long, J. S., & Ervin, L. H. (2000). Using heteroscedasticity consistent standard errors in the linear regression model. The American Statistician, 54(3), 217–224.

• Cohen (1988)

  Method

  The power analysis behind the recruitment target, and the guidance that a pretest covariate correlated with the outcome buys back part of the sample size an unadjusted comparison would need.

  Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

• Kulik & Fletcher (2016)

  Method

  One of the three meta-analytic sources the target effect size is read off, rather than being assumed or borrowed from a study with a no-treatment control.

  Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42–78. https://doi.org/10.3102/0034654315581420

• Ma et al. (2014)

  Method

  The second of those sources, and the closest in design to this comparison: intelligent tutoring against other instructional methods rather than against nothing.

  Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901–918. https://doi.org/10.1037/a0037123

• VanLehn (2011)

  Method

  The third, and the reason the target sits at the lower end of the tutoring literature: this study compares two active instructional conditions, which is the comparison that produces the smaller effects.

  VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221. https://doi.org/10.1080/00461520.2011.611369

• Shadish, Cook, & Campbell (2002)

  Method

  The reason the comparison is described as a treatment package. The conditions differ in scaffold assignment and in how incorrect-answer feedback is produced, so an effect belongs to the package, not to fading alone. That limit is stated rather than glossed.

  Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

• Schulz, Altman, & Moher (2010)

  Method

  The reporting standard: a participant flow diagram, exclusions counted by reason, effect sizes with intervals, and primary against sensitivity results side by side.

  Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMJ, 340, c332. https://doi.org/10.1136/bmj.c332