Research
I study how AI systems should hand help to people: how much support to give, when to take it away, and on what evidence. I finish an MS in Human Centered Computing at the University of Nebraska Omaha in December 2026, and the thesis below is the current work. The protocol, the instruments and the multi-agent system that drives the adaptive condition are built and running; the study is awaiting University of Nebraska Omaha Institutional Review Board approval before any participant data is collected.
Current Research
ORCID https://orcid.org/0009-0000-7922-2009. No works are registered against it yet, which is what a persistent identifier looks like before a first publication.
The Effects of AI-Driven Adaptive Scaffolding vs. Static Worked Examples on Cognitive Load and Skill Acquisition
Does an AI system that adjusts how much help it gives, based on how the learner is actually performing, produce lower extraneous cognitive load and better skill acquisition than a fixed schedule of worked examples that fades on a timetable? Every AI product that offers assistance makes this call. I built an experiment that measures it.
Method and instrument
The design is published in full as a study protocol: conditions, scaffold levels, tasks, measures, and how the adaptive condition decides. Every number on that page is generated at build time from the code the study runs, so it cannot describe a version of the study that is no longer the one being run. The item bank stays withheld until data collection closes, and the protocol says why.
Design
- Between-subjects, two conditions, random assignment at consent.
- Adaptive: the support level for each task is chosen from the learner's recent accuracy.
- Static faded: a fixed schedule, level 3 then 2 then 1, regardless of performance.
- Three Python for loop tasks: plain iteration, filtering on a condition, and accumulation.
- Three support levels: a complete worked example, partial code with the key line blanked, and the prompt on its own.
Instrument
- Nine screens: consent, demographics, pretest, learning intro, learning activity, cognitive load, posttest, post-study questionnaire, debrief.
- Pretest: three multiple-choice items, used as the prior-knowledge measure.
- Posttest: eleven items, ten multiple choice plus one coding challenge.
- The coding challenge carries no answer key. The server stores the submission ungraded and it is scored afterwards against a five-point rubric (syntax, indentation, variable use, executes, correct output). Code is executed during the learning activity, not on this item.
Cognitive load measurement
- The Klepsch et al. (2017) differentiated scale, 9 items on a 1 to 7 response range.
- 3 intrinsic, 3 extraneous, 2 germane, plus 1 embedded attention check.
- Subscale means are computed server-side, and the schema enforces the exact item count per subscale, so a malformed submission is rejected instead of quietly scoring.
Process measures
- Per task: keystrokes, backspaces, pauses longer than fifteen seconds, help requests, and time to first keystroke.
- Per phase: start and end timestamps, tasks completed, and whether the phase auto-advanced on its eight-minute cap.
- Post-study awareness items: whether the participant noticed the guidance changing, what they attributed it to, and whether the system felt responsive.
Research integrity
- Condition is assigned from a cryptographic random source, and the browser only ever receives an opaque group label. The condition names never reach the client, so the interface cannot leak the manipulation.
- Every submission is validated server-side: response times, non-empty responses, exact subscale lengths.
- Pilot and test runs are flagged in the database so they cannot mix into the analysis set.
- If the multi-agent system fails for an adaptive participant, the session ends and the record is flagged. It never falls back to the static schedule, which would file that participant under the wrong condition.
Hypotheses and analysis plan
Fixed before recruitment opens, which is the point of stating them here rather than after the data arrives. The primary cognitive load outcome is the extraneous subscale mean, the primary learning outcome is the posttest multiple-choice score, and pretest score is the covariate. The full plan, with the analysis populations and the reference list behind each decision, is on the study protocol.
The AI-adaptive condition will produce lower extraneous cognitive load than the static faded condition.
Test: One-sided Welch independent-samples t-test.
The AI-adaptive condition will produce germane cognitive load that is not meaningfully lower than the static faded condition.
Test: One-sided Welch independent-samples t-test. Secondary: TOST equivalence test, bounds d = plus or minus 0.30.
There will be no practically meaningful difference in intrinsic cognitive load between conditions.
Test: Two-sided Welch independent-samples t-test. Secondary: TOST equivalence test, bounds d = plus or minus 0.30.
The AI-adaptive condition will produce higher posttest performance than the static faded condition, after adjusting for pretest performance.
Test: ANCOVA with heteroscedasticity-consistent (HC3) standard errors, regressing posttest score on condition and pretest score. Secondary: Welch t-test on the raw posttest score, plus descriptive gain scores.
Within the AI-adaptive condition, scaffold level will fall as task position advances, which is what contingent fading looks like in the decision trace.
Test: Cumulative-link mixed model with participant-level random intercepts, if sample size permits. Secondary: Pre-specified fallback if the model is unstable: Spearman association plus a descriptive transition analysis of fade, maintain and increase after correct and incorrect responses.
Sample size and power
- Powered to detect d = 0.40 at 80% power, which needs about 100 participants per group, 200 in total.
- H1 to H3 are one Bonferroni-corrected family at alpha = .0167 per test, so holding that power takes about 120 per group, 240 in total.
- The recruitment target is a minimum of 200 participants, with 240 as the preferred goal.
- Equivalence bounds for the two equivalence tests are d = plus or minus 0.30.
Outcomes and covariate
- Primary cognitive load outcome: extraneous_load_mean. Mean of the extraneous items on the cognitive load instrument.
- Primary learning outcome: posttest_mc_score. Sum of correct responses across the ten multiple-choice posttest items, 0 to 10.
- Secondary outcome: germane_load_mean. Mean of the germane items.
- Manipulation check: intrinsic_load_mean. Mean of the intrinsic items.
- Covariate: pretest_score. Sum of correct responses across the three pretest items, 0 to 3. Entered as the covariate in the H4 model.
- Exploratory: Coding challenge rubric score. Scored offline by human raters against five binary criteria, 0 to 5, and reported separately from the confirmatory tests. It is not auto-scored, so it is not part of H4.
Analysis populations
- Randomized sample: Everyone with a recorded condition assignment. Used for CONSORT-style accounting and attrition reporting, not for hypothesis tests.
- Primary confirmatory sample: The randomized sample less the pre-specified exclusions below. Every confirmatory test runs here.
- Sensitivity sample: Keeps attention-check failures and sessions under fifteen minutes, while still excluding test data and genuinely missing outcomes. Every test is re-run here and reported alongside the primary result, so a conclusion that depends on the stricter screen cannot pass as one that does not.
Pre-specified exclusions
- Pilot and test runs, which the database flags at write time so they cannot mix into the analysis set.
- Sessions terminated by an adaptive-system failure. The session ends and is logged as a technical termination rather than falling back to the static schedule, which would file that participant under the wrong condition.
- Participants who did not reach the outcome the hypothesis under test needs.
- Participants who failed the embedded attention check.
- Sessions under fifteen minutes in total.
H1 to H3 are one cognitive load hypothesis family and each is tested at a Bonferroni-adjusted threshold. H4 is the primary learning outcome and is tested at the conventional threshold, outside that family. H5 is a pre-specified mechanism analysis carrying lower inferential weight, given that each participant contributes three scaffold decisions.
What the design rests on
Each of these is here because something in the experiment exists because of it. The full reference list, including the methodological sources behind the tests and the reporting standard, is on the study protocol.
Sweller (1988)
The reason cognitive load is the outcome rather than a covariate. Working memory is the binding constraint on schema construction, so an instructional change is measured by what it does to the load it imposes, not only by a score at the end.
Sweller, Van Merriënboer, & Paas (1998)
The three-way split of load into intrinsic, extraneous and germane, which is the structure of the measurement and of H1 to H3: extraneous is what adaptive support should reduce, germane is what it must not suppress, and intrinsic is the manipulation check.
Kalyuga, Ayres, Chandler, & Sweller (2003)
Why support is withdrawn on evidence of competence rather than on a clock. Guidance that helps a novice becomes redundant once a schema exists, and processing it then costs working memory. That is the failure the adaptive condition is built to avoid and the fixed schedule is built to expose.
Wood, Bruner, & Ross (1976)
The definition of scaffolding the whole manipulation inherits: control of the parts of a task that exceed a learner's current competence, withdrawn as competence grows. The three support levels are that idea made discrete enough to assign and to log.
Vygotsky (1978)
The zone of proximal development, which is the band the scaffold levels are meant to keep a learner inside and one of the three checks the antagonist agent argues from when it objects to a proposed level.
Sweller & Cooper (1985)
The worked-example effect, which is why the comparison condition is faded worked examples rather than unguided practice. The control has to be a treatment that works, or a difference in favor of the adaptive condition says nothing.
Renkl (1997)
The observation that the right moment to fade differs between learners, because self-explanation quality does. That individual variation is the gap a fixed 3-2-1 schedule cannot close and the adaptive condition is testing.
Shin et al. (2025)
Fading as the bridge between studying a solution and generating one, in a programming context. It is the reason the levels run complete example, partial code, prompt alone rather than switching support off in one step.
Faber et al. (2024)
Contingent, performance-based fading as the thing being tested, and the warning that comes with it: a fixed schedule can match a learner's needs by coincidence often enough to erode the difference. The fixed schedule here is strictly non-responsive so that coincidence is at least measurable.
Bjork & Bjork (2011)
The constraint on the whole adaptive design: performance during learning is a poor proxy for learning, so a system that minimizes difficulty is not the same as a system that helps. The antagonist agent exists to argue against over-support for this reason.
Bjork & Bjork (1992)
The account of retrieval and storage strength behind desirable difficulties, and the reason H2 is framed as germane load not being suppressed rather than as load being reduced across the board.
Zimmerman (2000)
The cyclical model of self-regulated learning that the adaptive condition externally enacts: the system monitors and adjusts on the learner's behalf, then hands that responsibility back as support fades.
Zimmerman (2008)
The methodological case for measuring regulation through process traces rather than self-report alone, which is why every scaffold decision, keystroke count and pause is recorded alongside the survey.
The OWA Architecture
The adaptive condition's decisions come from three agents with separate jobs, each running on a different model family so that no single model's habits decide how much help a learner gets. I designed it and I built it; it is what runs behind the study. The pattern itself, including the role contracts, what it costs to run and how it degrades, is specified on the OWA reference page. What follows is how this study uses it.
Worker
- Reads the learner's recent accuracy and proposes the next support level, with its reasoning.
Antagonist
- Argues against the proposal from a learning-theory knowledge base: cognitive load theory, the zone of proximal development, and the fading and expertise-reversal literature. It objects when support would come away faster than schema development justifies.
Orchestrator
- Makes the final call and records the rationale. It is the only agent whose output sets the level the learner sees.
What every decision records
- Previous level, final level, and direction: fade, maintain, increase, or initial.
- The antagonist's objection, the orchestrator's rationale, and the accuracy window the decision was computed from.
- Decision latency and which model actually served each role, so a provider fallback shows up in the data.
- The worker's proposal is kept as free text alongside the decision, so a proposal the orchestrator overrode is still in the record.
Failure handling
- Enforced output schemas on the worker and the antagonist, and a named fallback model for every role. The orchestrator is prompted for raw JSON instead, which is what the deterministic rule below exists for.
- If the orchestrator's response cannot be parsed, a deterministic rule sets the level and the decision is marked as a fallback, so those cases can be analyzed separately instead of passing as model output.
- Every failure writes an incident row: which agent, error type, attempt number, and whether the session was terminated.
Teaching
Three audiences for the same material: students at UNO, practitioners picking up new tools inside a company, and people I have managed.
Guest lecturing
- Invited guest lectures at the University of Nebraska Omaha in ISQA 8460, Internet of Things (IoT), Big Data and the Cloud, a three-credit graduate course taught by Professor Sachin Pawaskar, Ph.D. Most recently in 2024.
- The sessions cover IoT, big data and cloud technology applied in the transportation industry: what the platforms are actually used for, and what breaks at fleet scale.
- The full speaking and lecture record is on Ideas & Impact.
Practitioner enablement
- Onboarded 300+ developers, product managers, and IT staff to AI coding tools.
- Worked with technology leads across 20+ product engineering teams to shape AI-enablement strategy.
Mentoring
- People I have managed and coached have gone on to roles including AI Product Manager and Executive Director of Technology.
Where this goes next: I want to keep working on this line of research in a PhD program. That is a goal I am working toward, not a program I am enrolled in.
Broader Research Interests
Areas I'm exploring as part of my academic work or future research directions.
Cognitive Load in Human-AI Systems
Adaptive Learning Interfaces
Trust & Mental Model Alignment
AI Code Quality & Developer Efficacy
Human Factors in Technology Adoption
Participate in Research
Your participation is anonymous and helps advance knowledge in human-computer interaction and learning sciences.
A controlled study of how AI-driven adaptive scaffolding affects learning efficiency and cognitive load, comparing dynamic scaffolding against a fixed faded schedule on Python for loop tasks. The protocol, instruments and multi-agent scaffolding system are built and the study is awaiting Institutional Review Board approval before any participant data is collected.
Design: Between-subjects, two conditions
Status: Awaiting IRB approval, not yet recruiting
How I protect you:
- All responses are anonymous
- Participation is completely voluntary
- You may withdraw at any time
- Data used for academic research only
- No personally identifying information collected
A submission is under review at the University of Nebraska Omaha Institutional Review Board. The protocol number will be posted here on approval, and no participant data is collected before then.
Questions about your rights as a research participant go to the UNO IRB at (402) 554-2393 or unoirb@unomaha.edu, not to me.
Questions about the study itself? Contact me.
Common questions
What does Tristan J. Nolan's MS thesis investigate?
Tristan J. Nolan's MS thesis at the University of Nebraska Omaha studies the effects of AI-driven adaptive scaffolding versus static worked examples on cognitive load and skill acquisition: whether an AI system that adapts its level of help to a learner's performance beats fixed, pre-prepared examples. The study is offline pending IRB approval and is not currently recruiting participants.