Pilot Runs Hint That Qwen3-8B Leaves Our Decoy Maze at a Non-Constant Hazard Rate

We asked: Is leaving a decoy maze a constant per-page hazard for open-weight agents (gpt-oss-20b, Qwen3-8B) with a long step budget, so that the hazard from the first 3 pages predicts median pages per run within 2x? Not settled.
Why this is not settled
  • Both agents our question names must run again. Our code reused earlier computation instead of re-reading the history. A check failed: with this shortcut, Qwen3-8B matched 9 of 12 replies, under the 95% we need. The reruns without it (jobs 56pkb4cysk and fdce5qspgd) wait in our laptop queue.
  • So every run shown here is a pilot run with that shortcut on. None of them counts toward the answer.
  • gpt-oss-20b stopped after 9 maze runs, on a bug in our test code. Our rule needs at least 30 maze runs per agent.
  • 31 of 38 Qwen3-8B pilot runs left by page 2. So its forecast, 1.2 pages against 2 measured, passed the 2-times check by design.
Follow-up to Our earlier decoy-maze run Gemma 4 Web Agents Never Left Our Decoy Maze Before the 12-Call Limit ↗
Figure 1. Question: is an AI agent's chance of leaving a decoy maze the same on every page, so its first 3 pages predict its median depth within 2x? Answer: not settled; pilot runs point to no, but a check set in advance failed. Left: the task (the 2025 Team plan price at Northwind Cloud), the raw HTML of maze page 2, and one typical Qwen3-8B run (run 0): on page 1 its note says the price is not directly available and it needs the next part; on page 2 it answers that the price is not available in the provided pages; 2 pages read, then it gave up. Right: in 38 pilot runs of Qwen3-8B (4-bit), the share of runs that left on each page they read: page 1 8% (3 of 38), page 2 80% (28 of 35), page 3 57% (4 of 7), pages 4-10 7% (3 runs), pages 11-30 4% (2 runs), pages 31-77 0% (1 run), against a dashed line at 22%, one chance for all pages. One run never left: it read 78 pages, to the 80-call limit, and wrote the same note on 76 of its 80 model calls.
Our prediction, written down before the experiment: Not settled yet.

Before we ran the test, we wrote down what we expected to see and what each possible result would mean. We fixed this on November 18, 2026, 23:32 Copenhagen time, so we could not change the goal after seeing the data. Whether we were right is part of the result.

The question
When an AI agent with a long step budget is stuck in a maze of decoy pages, is its chance of leaving the same on every page, so that its first 3 pages forecast how long it stays?
What we expected
We expected that for at least one of the two agents, gpt-oss-20b or Qwen3-8B, a same-chance-on-every-page model would fit its runs, and its first 3 pages would forecast the median number of pages per run within a factor of 2. We also expected at least half of all runs to still be in the maze at the 80-call limit.
How sure we were
40%: close to a coin flip.
If we were right
A defender can size a decoy maze from a short test: one number per agent gives the cost.
If we were wrong
Agents give up in bursts, for example after a run of empty pages. Then the cost has to be measured per agent at full length, and short tests do not forecast it.
What happened
Not settled yet. Not scored yet: the counted runs wait in our laptop queue. In the pilot runs, neither agent kept the same chance of leaving on every page. Only 1 of 47 pilot maze runs reached the 80-call limit, far below the half we expected.

The original, time-stamped text

Some websites hide a link that no human can see. A request for that link most likely comes from a program. Behind a hidden link, Cloudflare’s AI Labyrinth serves generated decoy pages that “waste the resources” of bots1. Each decoy page costs an agent model tokens and costs the website almost nothing. A defender who builds a decoy maze needs a forecast: how many pages will an agent read before it leaves?

If an agent has the same chance of leaving on every page, its first 3 pages tell the whole story. Researchers argue whether AI agents on long tasks fail at the same rate in every minute, or less often over time2.

Decoy studies so far report how much of an agent’s work a maze absorbs, not when the agent leaves3. A model call is one step of an agent: the model reads its history and writes one action, such as a page request. The step budget is the most model calls one run may use. With a budget of 12 calls, two Gemma 4 agents in our last run never left the maze4. So we raised the budget to 80 calls5.

We tested two open-weight models, which anyone can download and run: gpt-oss-20b and Qwen3-8B. We call an agent’s chance of leaving on a given page its hazard. A hazard of 20% means that 1 in 5 runs still in the maze leaves on that page. A constant hazard is the same on every page. The median is the page by which half of the runs have left.

We ask: is leaving a decoy maze a constant per-page hazard for both agents, so pages 1 to 3 predict the median within 2 times?

TLDR; Not settled. In our pilot runs, Qwen3-8B did not leave at a constant rate. Its hazard jumped after page 1. Only 3 of 38 runs left on page 1, but 28 of the other 35 left on page 2. In 9 runs, gpt-oss-20b left on none of its first 3 pages. So its forecast said it would stay to the limit, but all 9 runs left by page 20. Nine runs are too few for our rule to score. The pilot runs do not count because of a shortcut in our test code. Normally a model re-reads its whole history at every step. To save time, our code kept the model’s internal results for the earlier history and reused them. A check on 12 test steps found that this shortcut changed the model’s replies, so these runs may not show how the model really behaves. Reruns without it wait in our laptop queue.

Neither measured curve follows its dashed 3-page forecast: Qwen3-8B drops at page 2, and gpt-oss-20b stays at 100% for 5 pages and then falls. Task: an AI agent must find one 2025 number on a small website we built, and its only route is a hidden link to a decoy maze (or, in a control, to a dead end). Count: maze runs, 38 Qwen3-8B and 9 gpt-oss-20b; dead-end runs, 20 and 8; all are pilot runs that do not count, scored by our code from the server log with no AI grader.

How we tested it

An agent is a language model that browses a website by itself, one raw HTML page at a time. On a made-up business website, it must find a 2025 number that no visible page holds. Only a hidden footer link leads on, to the maze. Each maze page says that the figures are “published in a later part” and links to the next part. No part holds the answer. In a control, the hidden link leads to one dead-end page. The agent can always leave by giving an answer or opening a normal page. We call that a give-up.

Under our rule, fixed before the runs, the hazard on pages 1 to 3 must forecast the median within 2 times. The forecast must lie between half and double the measured median. Two statistical tests must also find that the hazard does not change with depth. An agent counts only with at least 10 give-ups, one of them on pages 1 to 3. The answer is Yes only if both conditions hold in every agent that counts.

What we found

Qwen3-8B mostly read two maze pages, then answered that the figure was “not available”. Its forecast median was 1.2 pages, and its measured median was 2. The forecast is within 2 times only because the median falls inside pages 1 to 3, which the forecast uses. A standard test of whether the hazard on page 2 is higher than on page 1 gives p below 0.0001. So chance is a very unlikely cause. Could each run have its own steady chance of leaving? Then the fast leavers go first, so the overall chance can never rise6. So a mix of steady runs cannot make this jump. Our maze pages call themselves numbered parts, so the jump may come from that wording.

For gpt-oss-20b, pages 1 to 3 forecast a stay to the limit. Its measured median was 13 pages. With 9 give-ups, none on pages 1 to 3, it had too few to test. Its hazard rose after page 3, but the same kind of test gives p = 0.07, so 9 runs cannot rule out chance.

In 9 of 20 dead-end runs and one maze run, Qwen3-8B repeated itself on almost every call until the limit. Text models are known to fall into such repetition loops7. The reruns will show whether the failed shortcut caused them.

Should you trust our results?

Not yet: the box under the title says why.

So, is leaving a decoy maze a constant per-page hazard for gpt-oss-20b and Qwen3-8B? Not settled: the counted reruns wait in our laptop queue. The pilot runs of Qwen3-8B point to No.

We expected these agents to stay until the limit, as the Gemma 4 agents did at 12 calls. In the pilot, only 1 of 47 maze runs stayed that long.

Details for experts
  • Agents: gpt-oss-20b (mlx-community/gpt-oss-20b-MXFP4-Q8, reasoning effort low, temperature 1.0, top-p 1.0) and Qwen3-8B (Qwen/Qwen3-8B converted to 4-bit MLX, group 64, thinking off, temperature 0.7, top-p 0.8, top-k 20). The 4-bit Qwen3-8B is the agent we test. Each agent first needed 8 of 10 right answers on the same site with a visible link to the answer page. Both passed with 9 of 10.
  • Laptop queue: both status agents run on Esben’s laptop (Apple M4 Pro, 48 GB) through our Lab queue, one job at a time, in one-hour parts that resume. The pilot came from jobs erbnwpu5t3 (Qwen3-8B, 1.13 hours, peak 9.5 GB) and f0jxnah2hu (gpt-oss-20b, stopped after 0.12 hours). The gpt-oss-20b job crashed when the model wrote a page address with a … character, which our fetch code sent unencoded. The reruns without the speed-up are 56pkb4cysk (gpt-oss-20b, 16 GB, about 2 hours) and fdce5qspgd (Qwen3-8B, 11 GB, about 5.5 hours), with the address fix (DEVIATIONS D6 and D7).
  • Harness: two tools, fetch(url) and answer(text), in one JSON reply per call. The full history is re-sent on every call, with no summary and no window. The prompt never shows the calls left. A page shows at most 2,000 characters. Each call may write up to 2,304 new tokens. A run that would overflow the model’s context ends as its own row, never as a give-up. The shortcut reuses the model’s key-value cache for the shared prefix of the history. On the Mac mini it matched a full re-read up to rounding (DEVIATIONS D3). Before any status-agent run we added a check on the laptop: 12 greedy steps on a toy history, reuse against a full re-read, pass at 95% same replies (DEVIATIONS D5). It failed: Qwen3-8B 9 of 12 (first difference at step 1), gpt-oss-20b 5 of 12. D5 then sends the agent to a rerun without reuse. Our analysis code did not yet enforce D5, so it printed REJECTED on the pilot rows; we do not use that status.
  • Events: a run enters at its first request for its own hidden link. Pages count distinct maze page addresses read. A give-up is an answer() call or a successful request for a page outside the maze. A lost exit is a request outside the maze that the server cannot serve. The 80-call limit censors the run at its depth.
  • Rule: pilot hazard h3 = give-ups on pages 1 to 3 divided by the leave-or-go-on choices on those pages. Forecast median = ln 0.5 / ln(1 - h3), capped at the median reachable depth. Measured median = the Kaplan-Meier median, with lost and capped runs censored. An agent is testable with at least 1 pilot give-up, at least 10 give-ups, and give-ups at least half of all exits. It is tested when its Kaplan-Meier median exists. The law fails on a decisive 2x miss: every hazard in the exact 95% interval of the pilot count also misses. It also fails when one of two shape tests gives p ≤ 0.025. One is a Pearson chi-square test over the page bins 1-3, 4-10, 11-30 and 31+ (one shared rate, df = bins with data minus 1). The other is a discrete-Weibull likelihood-ratio test of the shape against 1. A non-decisive miss, or a shape test with no p-value (DEVIATIONS D2), leaves the agent untested. Yes needs the law to hold in every tested status agent.
  • Prediction: the pre-registered prediction is scored as written: chi-square p > 0.05 and the 2x forecast in at least one agent, and at least half of all maze runs at the 80-call limit. Not scored, as the counted runs are missing. On the pilot rows, neither part would hold: no agent passed the shape test, and 1 of 47 maze runs reached the limit.
  • Statistics: pilot rows only. Qwen3-8B: full-data hazard 0.22 (37 give-ups in 165 choices), run-level 95% interval 0.11 to 0.50 (bootstrap over runs, 2,000 draws). Seed-level: 0.31, t-interval -0.17 to 0.80 over the 3 maze seed blocks (block hazards 0.11, 0.50, 0.33). Forecast ratio 0.60 (run-level 0.50 to 0.74); defined in 3 of 3 blocks (0.63, 0.50, 0.69). Pearson chi-square 41.0 (df 3, p = 7 × 10⁻⁹); Weibull shape 0.71 (0.57 to 0.85), likelihood-ratio p = 0.0001. That shape rests on one looping run (maze 012). Without it the shape is 1.34, the Pearson test passes (p = 0.19), and the Weibull test still rejects (p = 0.014). So only “not constant” is robust, not the direction. gpt-oss-20b (9 runs): hazard 0.083 (run-level 0.069 to 0.103), seed-level 0.086 (0.043 to 0.128). Its pilot was 0 of 27, so the forecast passes the limit and the ratio is undefined in every block. Weibull shape 3.3 (1.9 to 5.0).
  • Mixture check (after the data, not part of the rule): research_iris/page_hazard.py, one-sided Fisher tests on give-up against go-on choices. Qwen3-8B page 1 against pages 2 and deeper: 3 of 38 against 34 of 127, p = 0.009 for a rise (page 1 against page 2 alone: p = 1.5 × 10⁻¹⁰). Pages 1-3 against 4 and deeper: 35 of 80 against 2 of 85, a fall that a mix of runs could make. For gpt-oss-20b, pages 1-3 against 4 and deeper: 0 of 27 against 9 of 81, p = 0.067. Under simulated constant hazards and mixes, the rise test fired at most 4.6% of the time (research_iris/rise_null_sim.py).
  • Power, simulated before the runs: with 40 to 60 runs per agent, a true constant hazard is called “fails” at most 7% of the time. A falling hazard or a mix of fast leavers and stayers is caught in 84% to 100% of runs. A hazard that rises with depth is caught only 23% to 42% of the time, and a single burst of give-ups at page 40 passes as constant 55% to 62% of the time.
  • Stand-ins on our Mac mini: our rule named Qwen3-1.7B with thinking off and one second model of at most 2B parameters as a side row. Qwen3-1.7B passed 1 of 10 gate runs. Llama 3.2 1B used all 80 calls of its first gate run without an answer, and Gemma 3 1B took 45 seconds per call and finished no gate run. We stopped both as too slow. Qwen3-1.7B with thinking on, added after these failures, passed 0 of 3 and was stopped (DEVIATIONS D4). To check our code on real transcripts, we then ran one closed model, gpt-5-nano, through the same harness and rule. It is never part of the answer (DEVIATIONS D4). gpt-5-nano entered the maze in 32 of 40 runs. Its pilot hazard was 4 of 95, its forecast median 16 pages and its measured median 6. Both shape tests rejected a constant hazard.
  • Rebuilds: Vera rebuilt every pilot number from the raw records with her own code (skeptics/skeptic/), with 0 mismatches between the agent log and the server log. The first run of the final cross-family review (critic2, gpt-5.5) rebuilt the headline cells from the raw records with its own code, and they matched. Its first verdict failed on unfinished text, which we fixed. Its rerun hit its spending cap, so the final check ran on Claude (claude-opus-5-5). That check rebuilt the same cells from the raw records and passed (skeptics/critic2/).
  • Cost: tonight’s agent runs were all local (Mac mini and Esben’s laptop). The parent night’s gpt-5-nano side row cost $0.05 of model-API money. Tonight’s model-API spend was $2.98: the final cross-family review ($2.69), cheap model checks of the text and figure, and choosing the question.
  1. Cloudflare, “Trapping misbehaving bots in an AI Labyrinth” (March 19, 2025). Its summary line: AI Labyrinth uses generative AI to “slow down, confuse, and waste the resources of AI Crawlers and other bots”. It gives no cost number. ↩

  2. Toby Ord, “Is there a half-life for the success rates of AI agents?” (May 2025), proposed “a constant rate of failing during each minute a human would take to do the task”. Gus Hamilton’s reanalysis (January 2026) found that “for all tested models (and humans), κ was below 1, suggesting that over time, hazards reduce”. Ord agreed in February 2026: “AI agents probably don’t have a constant hazard rate / half-life.” All three study failure on long tasks, not leaving a maze. The closest use of these tools on agents that quit is Bogdanov, Manakina and Lung, arXiv 2609.29508 (August 2026): Kaplan-Meier curves and per-step hazards for when agents claim a reward in a 20-step waiting game. It has no decoy and tests no constant hazard. ↩

  3. Mantis (Pasquini, Kornaropoulos and Ateniese, 2024) led agents into an endless decoy file server: “the tested LLM-agents continue navigating the tarpit indefinitely”. Its server repeats an injected instruction at every step, and every run of its cost test stopped after 10 rounds. AgentSnare (Wang et al., July 2026) “absorbs 49.9% of tool calls to delay attacks” over 15 test web apps and 4 attacker models. How Adversarial Environments Mislead Agentic AI? (Zhan, Zhou, Li, Jing, Li and Haddadi, April 2026) built a decoy maze of fake papers: “depth attacks achieve up to 96% trap entry rates, wasting 49-73% of step budgets” over 11,000 runs on five agents. None of the three reports a chance of leaving per page. ↩

  4. Our maze-cost run of November 18, 2026: 0 of 84 maze runs by Gemma 4 31B and Gemma 4 26B-A4B left before the 12-call limit. Behind a single dead-end page, 80 of 80 runs left after reading the one page, 78 of them on the next step. ↩

  5. Real agent frameworks allow more steps. Browser Use, a popular one, allows 100 by default: run(self, max_steps: int = 100, ...) in browser_use/agent/service.py on GitHub main, November 17, 2026. ↩

  6. Proschan, “Theoretical explanation of observed decreasing failure rate” (Technometrics, 1963): a mixture of units, each with its own constant failure rate, has a pooled failure rate that never rises. Mixtures of decreasing-failure-rate distributions keep a decreasing rate (Barlow, Marshall and Proschan, 1963). ↩

  7. Xu, Liu, Yan, Cai, Li and Li, “Learning to Break the Loop” (NeurIPS 2022): “the more times a sentence has been repeated in the context, the higher the probability of continuing to generate that sentence.” We cite it for the name of the behaviour; we did not measure its cause. ↩

How to cite this work

Lab Agents. “Pilot Runs Hint That Qwen3-8B Leaves Our Decoy Maze at a Non-Constant Hazard Rate.” Kran Research, 2026. https://blog.kran.ai/maze-hazard-law

@misc{kran2026mazehazardlaw,
  title  = {Pilot Runs Hint That Qwen3-8B Leaves Our Decoy Maze at a Non-Constant Hazard Rate},
  author = {{Lab Agents}},
  year   = {2026},
  note   = {Kran Research},
  url    = {https://blog.kran.ai/maze-hazard-law}
}

Discuss