Code Replay: Detect Copy-Paste Farms Without Slowing Hires

Its end of quarter. Your VP of Sales is pushing for a faster implementation cycle because two enterprise renewals hinge on on-time delivery. Recruiting celebrates a senior platform engineer hire: top score on the coding assessment, fast turnaround, confident answers. Three weeks later, Support is escalating P1s because deployments are breaking. Engineering leadership is quietly pair-programming with the new hire and realizing something uncomfortable: they cant reproduce even the simplest refactors without magic appearing in their editor. This is the revenue version of funnel leakage: you paid for a signal (assessment pass) that didnt correlate with Day 1 work. Replay is how you stop buying counterfeit signals.

Your anxieties are rational: - Speed: Any additional gate can slow time-to-offer and create candidate drop-off. - Cost: Bad hires dont just cost replacement; they consume scarce reviewer time, onboarding cycles, and incident bandwidth. - Risk: Fraudulent hiring is now a security vector, not just an HR mishap. In one real-world pipeline, 1 in 6 applicants to remote roles showed signs of fraud (Pindrop). - Reputation: If a customer learns you staffed a project with someone who misrepresented identity/skills, you get hit twice: delivery failure and trust erosion. Replay lets you keep the funnel moving while adding defensible evidence when something looks off. Youre not trying to turn your hiring process into surveillance; youre trying to keep the integrity of your signal-to-noise ratio high enough that downstream execution stays predictable.

A robust replay system does not require full screen recording. You want minimal telemetry that reconstructs the coding workflow while staying privacy-first: - Editor events: insert, delete, replace, selection changes (with timestamps) - Paste events: length, source type (keyboard/menu), and the pasted text or a hashed representation (depending on policy) - Execution events: run/test invoked, compile errors, unit test failures - File events: created/renamed, language mode changes - Environment events: disconnects, tab switches (if available), focus loss Pragmatic privacy guidance: - Capture just enough to produce an Evidence Pack and support appeals. - Use Zero-Retention Biometrics principles for identity steps (dont mix biometrics with code telemetry; keep concerns separated). - Encrypt at rest (256-bit AES baseline) and apply short retention for raw telemetry; keep only derived summaries longer if Legal prefers.

Replay data becomes useful when you translate it into a small set of measurable signals that correlate with farm patterns. Start with signals that are hard to fake and easy to explain in an audit: Important: allow open book resourcefulness. A candidate pasting a small helper snippet or boilerplate isnt fraud. The policy line should be about authorship transfer, not reference use.

Use a policy file so reviewers arent making ad hoc decisions and candidates get consistent treatment. This example routes only high-signal sessions to review and defines an appeal-safe Evidence Pack.

CRO reality: the moment you add a manual step, your funnel can stall. The fix is to treat replay review like a small, SLAd exception queue. Step-by-step: Operational guardrails: - Keep the reviewer pool small and trained to reduce variance. - Cap daily reviews per reviewer to avoid fatigue-induced errors. - Track false positives (appeals granted) as a first-class metric.

Two sessions can both end with a working solution. Replay shows whether they arrived there like an engineer or like a courier. Pattern A: likely farm injection - 610 minutes idle at start (no typing, no runs) - One large paste (hundreds/thousands of chars) into the main file - Minimal edits (rename variable, adjust import) - Single run; tests pass; done Pattern B: strong, open-book candidate - Starts with scaffold + comments (small typing bursts) - Pastes small snippets (e.g., regex, util) interleaved with edits - Runs tests multiple times; fixes errors iteratively - Refactors naming/structure near the end Your reviewers should be trained to look for work signatures: incremental construction, error correction, and local reasoning. Thats what correlates with Day 1 performance.

IntegrityLens AI is built for exactly this problem: turning fragmented hiring steps into one defensible pipeline: Source candidates  Verify identity  Run interviews  Assess  Offer. Heres how it connects to replay-based integrity signals: - ATS workflow: Manage candidates, stages, reviewer assignments, and audit trails in one placeno spreadsheet side-queues. - Identity verification (under 3 minutes; typically 23 minutes end-to-end for document + voice + face): Verify who is showing up before the interview starts, so replay findings arent blamed on identity uncertainty. - Fraud detection: Convert replay telemetry into integrity signals and route by Risk-Tiered Verification (auto-pass vs. step-up vs. review). - AI screening interviews (24/7): When replay flags risk, you can trigger an on-demand, structured follow-up interview without waiting for scheduling. - Technical assessments (40+ languages): Capture coding workflow signals across common stacks, not just one niche environment. - Evidence Packs + governance: Package replay evidence in a consistent format for Recruiting Ops, TA leaders, and CISOs to review and sign off. Who uses it: TA leaders for funnel health, Recruiting Ops for workflow control and consistency, and CISOs for fraud-as-a-security-vector governance.

Replay can be privacy-sensitive if implemented carelessly. The best operator posture is: record the minimum required, be transparent, and design an appeal flow. Key controls to bake in: - Notice + consent: Tell candidates you capture editor interaction telemetry to ensure assessment integrity, and describe retention. - Retention limits: Keep raw replay events short-lived; retain only Evidence Packs needed for disputes. - Access controls: Restrict replay access to trained reviewers; log access for audits. - Fairness checks: Monitor whether thresholds disproportionately flag certain groups or geographies (often driven by latency/disconnect patterns). - Explainability: Every escalation should cite which signals fired. The replay looked weird does not survive scrutiny.

• Where are we currently blind: final output only, or do we see workflow evidence? - Whats our reviewer fatigue rate (how many assessments get manual review, and why)? - Do we have a documented open book policy that distinguishes resourcefulness from authorship transfer? - Can we explain every reject decision with an Evidence Pack that Legal would defend? - If fraud increases next quarter, can we tighten thresholds without blowing up time-to-offer?