Algorithm Report

Civix Learning Algorithm Report

This document describes the learning algorithm in apps/mobile/lib/algorithm.ts: behavior, constants, and optimization status. It is maintained to reflect the implementation and tests in that file.

1. Algorithm version and USCIS constants

  • Version: ALGORITHM_VERSION = 5.4.0 (lib/algorithm.ts:2339) — first-question flow-zone sequencing across adaptive learning modes.
  • USCIS test format:
    • Standard: 128 questions in pool; 20 asked at interview; 12 correct required to pass.
    • 65/20 (senior): 20 designated questions; 10 asked; 6 correct required to pass.

Constants in code: USCIS.TOTAL_QUESTIONS, USCIS.EXAM_QUESTIONS, USCIS.PASSING_SCORE, USCIS.SENIOR_QUESTIONS, USCIS.SENIOR_EXAM_QUESTIONS, USCIS.SENIOR_PASSING.

1.1 Architecture — four cooperating layers

The engine is not "SM-2" alone; earlier revisions of this document described only the scheduling layer. Four layers cooperate, each answering a different question:

LayerQuestion it answersCode
SM-2 schedulingWhich questions are due to enter a sessionalgorithm.ts:380-426
Elo / 2PL flow zoneIn what order to serve them, livealgorithm.ts:466-506, :699-770
BKT (Bayesian Knowledge Tracing)How well is each item knownalgorithm.ts:549-570
Monte-Carlo simulationWhat is the probability of passingalgorithm.ts:623-648

Flow zone. Ordering targets an expected score of ~0.75, so difficulty tracks the learner. Ability and item Elo are clamped to [600, 2000] (:439-441) and the system self-stabilises: ability drifts until the learner's graded score matches expectation.

Pass probability is free-recall, not multiple choice. The simulation uses a guess floor of 0.03, not the MCQ 1/4 (:573-594). The real interview is oral recall, and a 0.25 floor inflated weak learners' odds roughly twofold. This is the single most important honesty decision in the file.

Exemption modes are simulated with the correct shape — draw 20 need 12, or draw 10 need 6 from the designated senior pool only (:650-683).

Determinism. The pure core contains no Date.now() or Math.random(). All shuffles take seeds, and the Monte-Carlo uses mulberry32 with a fixed seed, so a displayed readiness number only moves when mastery moves. Session seeds are time-based by design so sessions vary, while the pipeline stays seedable in tests.

Known characteristics (as of 2026-08-03)

Documented so they are not rediscovered as bugs. All of these lean pessimistic — displayed readiness understates rather than overstates, which is the safe direction for a paid product.

  • A correct answer at exactly the learner's own average speed grades quality 3 (:367-371); quality 4 requires beating their own EMA. Consistently-correct steady responders therefore plateau around BKT 0.75–0.85 rather than 0.99.
  • Ease factor does not decrease on failure (:390-399); it only falls via the slow-correct path. Leech detection (:834-836) and the streak-0 weak boost (:944-955) catch the same items.
  • "New" questions outrank overdue ones (10000 vs ~5000–8000, :901-934), so the bank is swept for coverage before first reviews come back. Deliberate for a finite exam bank; looser early spacing than strict SM-2.
  • BKT does not decay with time. getPassProbability reads raw bkt (:679-682), so a learner returning after a long absence sees unchanged pass odds even though calculateRetrievability (:2196-2217) knows better. This is the main outstanding truthfulness gap and the first planned algorithm change.
  • Two mastery definitions coexist: the practice grid uses lib/stats/mastery.ts:38-47 (streak ≥ 3), readiness uses statsV2.mastered (streak ≥ 3 AND seen ≥ 5 AND ≥ 80% accuracy, :824-832).

Readiness weighting caveat

lib/home-readiness.ts:361-370 composes the headline readiness score almost entirely from civics signals; the English test (reading/writing/speaking) enters only through interviewReadiness, weighted at 5% and itself averaged 4-way with civics. A learner who has never practised the English components scores a flat 60 on that component (home-readiness.ts:186). Since English is an independent way to fail the real interview, this under-weights a genuine failure mode. Tracked as a post-launch change.

2. Mastery levels

Each question is classified into one level via getMasteryLevel(stats):

LevelMeaningHow it's computed
newNever seen!stats or stats.lastSeen === 0
learningSeen but not yet learnedNot mastered, stats.correct < 3
reviewingIn review cycle, not yet dueHas nextReview in the future
dueDue for reviewnextReview > 0 and now >= nextReview and within 24h of due
overduePast duenow > nextReview + 24h
masteredFully masteredstats.mastered === true (streak ≥ 3, seen ≥ 5, correct ratio ≥ 80%)

Usage in selection: Priority scoring favors new (base 10000+), then overdue, due, weak (streak 0), weak topics, low ease factor, and deprioritizes mastered (with exceptions when deadline is near). Review mode filters to due questions when any exist; learn mode favors new/learning; weak mode favors low streak or ease < 2.0.

2.1 Competency milestones (4-tier model)

computeCompetencySnapshot(pool, stats, options) computes a learner competency snapshot from:

  • mastered percent
  • rolling accuracy
  • overdue ratio
  • weak-topic density
  • pass-risk distribution

Tiers:

  • foundation
  • developing
  • proficient
  • interview_ready

Snapshot includes score (0-100), confidence, reasons, next tier target, and points-to-next-tier.

3. Study modes

ModePurposeFilter / fallback
learnNew and learning questionsFilter: no stats or lastSeen === 0 or correct < 3. Fallback: full pool if fewer than count.
reviewDue reviewsFilter: nextReview > 0 and now >= nextReview. Fallback: full pool if no due questions.
weakWeak topicsFilter: no stats (count as weak) or streak < MASTERY.MIN_STREAK or easeFactor < 2.0. Fallback: full pool.
coverageEnsure all seenReserves ~30% of slots for unseen; rest by priority.
categorySpecific categoryFilter by category or tags.
senior65/20Filter to senior-only pool (senior65 or SENIOR_QUESTION_IDS) and prioritize evaluatable questions (state-dependent items require stateProfile). No fallback to non-senior pool.
examExam simulationHandled by selectExamQuestions: category round-robin, 20 (standard) or 10 (senior) questions; senior exam uses same evaluatable-first guard.

4. Priority factors (calculatePriority)

Order of application (conceptually):

  1. New questions: Base 10000; +500 easy, +250 medium; deadline critical/intensive adds 3000/1500.
  2. Recently answered: -10000 unless due/overdue (then -500).
  3. Overdue: 5000 + min(daysOverdue × 200, 2000) + 500 × urgencyMultiplier.
  4. Due: 3000 + 300 × urgencyMultiplier (within 24h of due).
  5. Weak (streak 0, wrong > 0): 2000; +500 if seen < 1 day ago; +1000 × urgencyMultiplier if deadline.
  6. Weak topics: Sum of (weakTopics[tag] × 100) × urgencyMultiplier.
  7. Low ease factor (< 2.0): (2.5 - easeFactor) × 500; +300 × urgencyMultiplier when deadline and not relaxed.
  8. Spaced timing: Bonus by days since last seen (e.g. 1–3 days +300; >7 days up to +400); deadline critical/intensive tightens windows.
  9. Difficulty progression: If doing well (correct ≥ 3, streak ≥ 2), +200 hard, +100 medium.
  10. Mastered: Penalty -3000 (relaxed) / -1500 (intensive) / -500 (critical); reduced if days since > 7 or deadline near and days since > 3.
  11. Pass-risk layer: Adds weighted risk score (wrong-rate trend, streak breaks, low ease factor, overdue days, low-quality recent answers, deadline pressure) to prioritize likely failure points.

Deadline urgency multipliers: relaxed 1.0, moderate 1.5, intensive 2.0, critical 3.0.

5. SM-2 and mastery constants

  • SM2: Initial ease 2.5, min 1.3; initial interval 1 day; graduating interval 6 days; easy bonus 1.3.
  • MASTERY: MIN_STREAK 3, MIN_SEEN 5, MIN_CORRECT_RATIO 0.8, TEST_READY_THRESHOLD 0.9 (90% mastered = test ready).
  • Quality (0–5): Used in SM-2; correct answers get 3–5 by response speed; wrong get 0–2.

6. Optimization status

  • Correctness: MasteryProgress.learning returns only the count of questions in the learning and reviewing levels (no double-count with due/overdue). Unit test enforces separate counts for new, learning, due, overdue, mastered.
  • Selection: All study modes (learn, review, weak, coverage, category, senior) have defined filters and fallbacks. Review mode prefers due questions when present; coverage mode reserves slots for unseen; exam mode returns exactly 20 (standard) or 10 (senior) when pool is sufficient.
  • Senior hardening: Senior mode now avoids unanswerable state-dependent items when stateProfile is missing (e.g., governor/senator prompts), while still including them when profile context exists. Store fallbacks for practice and exam preserve senior-only scope.
  • Pass-rate optimization: Priority now includes calculatePassRisk(...) to surface weak/overdue/high-risk items earlier with urgency-aware weights.
  • Deterministic scenarios: shuffleArraySeeded(...) and optional shuffleSeed in selection APIs enable reproducible question order for testing and simulation.
  • Recommendations: getStudyRecommendations(...) accepts { isSeniorMode?: boolean } and generates mode-aware copy when the user is in senior mode.
  • Performance: weightedShuffle pre-normalizes weights once (O(n)); each pick is O(remaining length). Total cost O(n × count) instead of O(n²) per call.

7. Related files