Looks Correct Is Not Is Correct: How BlankCode Grades Real Code in a Real Sandbox

Practice That Actually Sticks

Programming skill decays unevenly. Concepts stick. Syntax fades fast. Fluency – producing code without stopping to look something up – fades first. Rereading documentation feels like learning, but mostly it isn’t: it’s recognition, not recall.

BlankCode is built around a narrower, harder-working idea: fill-in-the-blank exercises on real code. Enough structure to orient you, with the parts that carry the actual knowledge removed – a type annotation, a lifetime, a synchronization primitive, a base case. It sits between multiple choice, which is too easy because it’s disguised recognition, and a blank editor, which is too hard when you’re rusty enough that you never get started. Get a blank wrong, and spaced repetition brings back exactly that gap, on its own schedule, until it’s not a gap anymore.

None of that works if the grading is soft. So BlankCode holds itself to one rule: nothing is graded by pattern-matching or by an AI judge reading an answer and deciding if it looks right. A hidden test suite runs in a real sandbox, on the language’s real toolchain, and either it passes or it doesn’t. Six languages, six real toolchains, no toy interpreter standing in for any of them.

That rule protects learners from a soft grader. Making it trustworthy end to end meant applying the same rule to the exercises themselves.

Verifying the Content, Not Just the Learner

Every exercise is, underneath the UI, a claim: this reference solution passes these hidden tests, and every wrong-but-plausible alternative fails them. That claim is easy to write and easy to get subtly wrong, because most content review reads the code rather than running it. A blank that looks correctly placed, a hidden test that looks like it covers the right case, a reference solution that looks like it compiles – all of that can check out by eye and still be false.

content:verify closes that gap. It doesn’t read an exercise. It takes the reference solution – the actual answer – and runs it through the same sandbox pipeline a learner’s submission would hit, against the same hidden tests, and confirms it passes. Then it runs the deliberately wrong alternatives through the same pipeline and confirms they fail. An exercise that clears content:verify is one where the grading claim has actually been tested, not just read.

Running it for the first time against the full exercise catalog earned its keep immediately. It caught exercises where the reference solution didn’t actually satisfy its own hidden tests once executed – a subtle mismatch between what the blank asked for and what the test checked, invisible on read-through and fatal for anyone attempting the exercise. And it caught AI-drafted exercises that looked complete – correct syntax, plausible test names, a solution that appeared to satisfy the prompt – but hadn’t actually survived contact with a real compiler or interpreter. Static review catches the shape of an exercise. Only running it catches whether it’s actually solvable.

That’s the standard now applied before anything ships: “looks correct” and “is correct” are different claims, and a platform that grades learners by execution has no business shipping content that was only ever verified by inspection.

Deleting 492 Lines Because the Bottleneck Was Never There

The same execution model that makes exercises trustworthy also simplified the infrastructure underneath them. BlankCode originally ran submissions through a job queue – a worker process that pulled work off a queue, ran it, and wrote back a result.

The queue existed for one reason: something had to find the work. But a submission takes two to twelve seconds to run in the sandbox, depending on the language’s toolchain – well within a single HTTP request. Once that was measured, the queue wasn’t solving a capacity problem. It was solving a discovery problem it had created for itself.

Removing it deleted 492 lines of code, along with everything the queue implied: a polling loop, a lease reaper for jobs whose worker had died mid-run, and an entire class of bug where a crashed worker left rows stuck in a state nothing would ever pick back up. What replaced it is inline execution: the request that submits a solution is the same request that runs it and returns the verdict. Fewer moving parts, fewer states to reconcile.

The instinct to reach for a queue is usually about scale that hasn’t arrived yet. Here, the real constraint – a request timeout well above actual execution time – made the queue pure overhead once someone measured it against the thing it was supposedly protecting against.

The Takeaway

Both stories point at the same practice: we execute everything we ship, rather than trusting that it reads well. A queue justified by “distributed systems usually need one” is an unverified claim about architecture. An exercise merged without ever being run against its own hidden tests is an unverified claim about content. Neither failure shows up by reading. Both show up the moment something actually runs and checks.

Frame every correctness claim as something to execute, not infer. Build grading and content verification on the same execution path, so neither can drift from the other. Prove behavior under conditions close to production – real toolchains, real timing, real failure modes. Transfer the discipline in writing, so the next feature ships with the same bar. That’s what a client should expect from an engagement with us: fewer surprises after ship, not because we guessed better, but because we ran the thing before telling anyone it worked.

Curious what deliberate, spaced practice across six languages actually feels like? Try BlankCode. If you want this kind of rigor applied to your own product, take a look at our projects and our services, or reach out.