Real-world work scenarios
Scenarios escalate from single-task execution to ambiguous, multi-stakeholder ownership. Each is delivered as a timed workplace simulation with realistic artefacts — tickets, incomplete requirements, legacy code, production constraints and stakeholder pressure.
Progression
| Level | Stage | Scenario type | Deliberately imperfect | Competency proven |
|---|---|---|---|---|
| L1 Execute | A | Well-specified ticket with clear acceptance criteria | Nothing — baseline competence | Follows standards, delivers to specification |
| L2 Diagnose | A–B | Defect report: system behaves incorrectly under specific conditions | Cause unknown; logs incomplete | Reproduces, isolates, fixes, regression-tests |
| L3 Optimise | B | Working system that violates a stated performance or cost budget | Constraint is binding; naive fixes break accuracy | Profiles, quantifies trade-offs, optimises to budget (N8121) |
| L4 Build | B–C | Model handed over as a notebook; must become a supported service | No specification, no tests, no error handling | Writes SRS/HLD/LLD, refactors, tests, deploys (N8122) |
| L5 Integrate | C | Multi-component AI feature spanning retrieval, generation and tools | Sources conflict; upstream data changes mid-build | Systems thinking, interface contracts, evaluation design |
| L6 Operate | C–D | Live incident: quality regression or outage in production | Under time pressure with partial information | Triage, mitigation, rollback, root cause, post-mortem |
| L7 Decide | D | Stakeholder requests a change trading quality against cost or timeline | No correct answer; requires defensible judgement | Evidence-based recommendation and written justification |
| L8 Own | E | Ambiguous business objective with no defined solution | Requirements must be elicited; scope must be negotiated | End-to-end ownership from ambiguity to defended delivery |
Standing simulation environment
A permanent simulated organisation runs for the full programme duration: a ticket backlog, a code repository with review requirements, staging and production environments, a rotating on-call roster from Stage C, scheduled releases, injected incidents, and role-played stakeholders (product owner, client, security reviewer, data protection officer). Learners accumulate a verifiable delivery history rather than isolated exercises.
Scenario per subject
Each subject carries its own workplace scenario.
You inherit an undocumented data-pull script that fails silently every third night. Reproduce the failure, add structured logging and typed interfaces, write regression tests that capture the bug, fix it, and raise a pull request with a written root-cause note.
A teammate's pull request breaks the build and mixes three unrelated changes. Review it professionally, request specific changes, help split the commits, and get the pipeline green without a direct push to main.
Business reports a revenue dashboard is wrong. Trace the discrepancy through the join logic, discover duplicate keys inflating totals, quantify the impact, correct the pipeline, and write the incident summary for a non-technical stakeholder.
A stakeholder claims a new feature lifted conversion by 12%. Determine whether the sample supports the claim, compute the confidence interval, identify the confound, and present a defensible verdict.
A nightly aggregation job has grown from 4 minutes to 90 minutes. Read the execution plan, identify the missing composite index and the accidental cross join, fix both, and document the before and after.
A model service works locally but returns 502 in the cloud. Work down the stack — container port binding, security group, IAM permission, secret injection — and produce a runbook so the next engineer resolves it in minutes.
The client changes a requirement mid-sprint. Assess the impact on scope, timeline and dependencies, update the SRS and risk register, and communicate the revised commitment in writing.
Your model scores 0.94 AUC offline and fails in production. Discover that a feature was computed using post-event information, rebuild the feature set with a strict point-in-time join, and requantify honest performance.
Production inference must run within 200 ms at p95 on a 4-core CPU instance with 8 GB RAM, but the current model averages 850 ms. Establish the baseline, profile to locate the bottleneck, apply quantisation and batching, verify accuracy remains within the agreed tolerance, document the change in the model card, and formally report the trade-off to the data-science team.
C03 Deep Learning
A training run plateaus at chance accuracy. Systematically eliminate causes — data pipeline, initialisation, learning rate, loss definition — and document the diagnosis path.
Data science delivers a notebook model that must go live in three weeks under a 99.5% availability target. Write the SRS and HLD, refactor the notebook into a tested, containerised, schema-validated service, load-test it, remediate the defects found, document it, and take it through code review and release approval.
A customer-facing summariser occasionally invents policy numbers. Build a reproducible failure set, add grounding and schema validation, define refusal behaviour, and demonstrate a measured reduction in fabricated fields.
An internal policy assistant answers confidently from outdated documents. Add document versioning and recency metadata, implement hybrid retrieval with reranking, enforce citations, and prove the improvement with retrieval and answer metrics on a fixed evaluation set.
An agent authorised to issue refunds must never act alone above a threshold. Implement permission scoping, a human approval gate, idempotent tool calls, cost caps and a complete audit trail; then demonstrate correct behaviour under retry and partial-failure conditions.
Leadership asks whether a prompt change may ship. Run the regression suite, separate retrieval from generation effects, quantify quality delta with confidence, report cost and latency impact, and issue a go/no-go recommendation with evidence.
Red-team an assistant that has tool access to a customer database. Find and document the exfiltration path through indirect prompt injection, implement layered mitigations, and re-test to show the bypass rate falls.
You are behind on a committed deliverable because an upstream dependency slipped. Renegotiate scope with the appropriate people in writing, inform affected colleagues, protect confidential client information in the communication, and record the revised commitment.