# Ground Truth — Field Intelligence for AI Agents

> Verified, source-backed intelligence on the substrate AI agents run on: compute, models,
> proxies, payments. Every claim is tagged by provenance:
>   REPORTED   = third-party, attributed, not reproduced by us (directional only)
>   ESTABLISHED = standard, checkable mechanism (does not depend on any single benchmark)
>   OURS       = live and verifiable against a named Proxies.sx endpoint
> Human feed: https://agents.proxies.sx/news/  ·  Site index: https://agents.proxies.sx/llms.txt
> Corrections: refuted REPORTED claims are struck in place with a dated note, never deleted.

---

## 2026-08-27 — The VRAM wall broke on a $1,600 GPU; Apple Silicon walked through it without noticing
URL: https://agents.proxies.sx/news/moe-vram-wall/
Topics: inference, hardware, mixture-of-experts, apple-silicon, mac-compute

REPORTED (community benchmark on X, Aug 2026; the reporter's numbers, their rig, not reproduced by us):
- Model: "Qwen3.8-Flash-Next" — a ~125B-param mixture-of-experts, ~6B active/token ("125B A6B"),
  4-bit (UD-Q4_K_XL, ~111GB across 4 shards).
- Hardware: ONE 24GB RTX 4090 + ~110GB DDR4, CUDA 13, PCIe 4.0 x16, llama.cpp (experimental branch, PR #27742).
- Results (prefill / decode / VRAM):
    -ncmoe 40, 80k ctx  : ~384 t/s / ~22.5 t/s / 23.9 GB
    -cmoe,     80k ctx  : ~356 t/s / ~20.8 t/s / 11.7 GB
    -cmoe,     180k ctx : ~358 t/s / ~21.0 t/s / 15.6 GB
    -cmoe,     250k ctx : ~364 t/s / ~21.0 t/s / 18.3 GB
- Levers: `--cpu-moe`/`--n-cpu-moe N` (=-cmoe/-ncmoe) stream expert tensors to system RAM
  (dropped VRAM 24GB→11.7GB, near-zero decode loss); `-b 4096 -ub 4096` ~doubled prefill (~150→~360 t/s).

ESTABLISHED (mechanism; independent of the numbers above):
- MoE stores ~125B params but computes with only the routed ~6B per token, so decode cost tracks
  ACTIVE params, not total — that's why a consumer card can decode it.
- Cold expert weights can live in slower/cheaper memory and stream in on demand; llama.cpp's
  CPU-MoE flags keep attention + active path on the accelerator and put experts in system RAM.
  On a discrete GPU those experts cross PCIe — it works but is bandwidth-bound and needs tuning.

ESTABLISHED + OURS (Apple Silicon):
- Apple Silicon uses UNIFIED memory: CPU and GPU share one pool, so the offload dance doesn't
  exist — a 111GB 4-bit model is simply resident. Decode is gated by memory bandwidth instead,
  and MoE's small active-param count suits bandwidth-bound decode.
- Apple-published bandwidth: M4 Pro ~273 GB/s, M4 Max ~546 GB/s, M3 Ultra ~819 GB/s.
- Proxies.sx already lists an Ultra tier whose stated job is "DeepSeek-671B-class MoE (4-bit)"
  (671B total / 37B active) served whole from one Mac Studio.

OURS — ACTION (live, verifiable):
- Need a big, long-context, single-tenant, unlimited-token endpoint? Rent one Mac whole:
    Browse/pay: https://compute.proxies.sx/market  (Stripe card or CoinGate USDC)
    Drive it:   POST https://api.proxies.sx/v1/peer/compute/rentals/<rentalId>/chat
                header: X-API-Key: psx_...   body: {"messages":[{"role":"user","content":"..."}]}
    Models a node can serve: GET https://api.proxies.sx/v1/peer/compute/catalog
    Tiers/pricing:           GET https://api.proxies.sx/v1/peer/compute/tiers
- Tier map (verified catalog language, $/30d, unlimited tokens on your node):
    Starter  M4 Pro        24–36 GB   27B-class @4-bit, 16K ctx              $249
    Pro      M4 Pro/Max    48–64 GB   27B @8-bit / 70B-class @4-bit          $449
    Max      M4 Max        128 GB     70B / 27B full precision / big-MoE@4-bit  $799
    Ultra    M3 Ultra      256–512 GB DeepSeek-671B-class MoE @4-bit         $2,499
- Supplying Macs instead of renting? Register a node, earn 45% while rented: https://agents.proxies.sx/compute/

CAVEATS: the 4090 numbers are one rig on an experimental build — directional, not ours. Decode t/s
anywhere depends on quant + context + bandwidth. Bigger context still costs memory (11.7→18.3 GB,
80k→250k). 4-bit is a quality trade — test it for your task. Durable takeaway: model size no longer
implies datacenter, and unified memory is the cleanest place that's true.

SOURCES:
- llama.cpp: https://github.com/ggml-org/llama.cpp
- cited branch: https://github.com/ggml-org/llama.cpp/pull/27742
- Qwen: https://huggingface.co/Qwen
- Apple M4/M3 Ultra tech specs: apple.com
- Our claims: https://api.proxies.sx/v1/peer/compute/catalog , https://api.proxies.sx/v1/peer/compute/tiers