The road to 60, all over again

The Old 3DS road

The 08-11 feasibility study said the port could not fit an Old 3DS (268 MHz, no L2 cache, 64 MB). It booted there on 09-29 at about 10 fps, raced at 60 in 2D on 10-02 and in 3D on 10-08. Every card was a physical SD-card trip.

Every card, on the calendar

A card is one round trip: build, ini, test plan, play session, log back. Hover or tap a card for what it carried.

1. 09-28 Shipped build (RDRAM 16) — Dies in the boot preload: [fatal] operator new failed size=175844 heapFree=32424. Console answers memtype=0, heap grant 24 MiB (25,165,824 B)
2. 09-29 First boot (RDRAM 8) — Boots and races. Wall p50 96.0 ms (p10 88.7, p90 111.3); GPU busy 3.6 ms; oth 37.3 ms. User: "it ran ! at like 9fps"
3. 09-29 C4 run 1: O3DS profile — GP wall 88 ms (11.3 fps), TT/menu 43.7 ms; aud 25.8 ms = audio-lock wait, confirmed on hardware
4. 09-29 C4 run 2: narrow audio lock — aud 25.8 → 0.2 ms; race wall 67.8 ms (14.7 fps), light scenes 40 ms
5. 09-29 Wave 2: HOME + audio SIMD — HOME freeze root cause isolated (bottom backlight session); 7/7 HOME resumes with screen on; audio_simd verify 391/0/0, 0 mismatches on silicon
6. 09-29 Wave 3: frameskip + O3DS preset — Logic 57–60 ticks/s (game at full speed for the first time); drawn 19–28 fps in races. User: "playable but nausea inducing". DrawProbe: 10.9 µs CPU per draw, 14 Mtri/s
7. 09-29 REL1 — Boot crash, deterministic. Four-title bisect: cold -Os crashes, stock and -O2+order boot
8. 09-30 REL2 — Boots. Race ~15 fps rendered (steady stuck at D=4), rw50 26–35 ms, "lots of awkward hitching": synchronous SD log flushes of 26–270 ms on main
9. 09-30 REL3 — "no hitches, steadier"; steady mostly D=3 (20 fps rendered), rw50 24–36 ms
10. 09-30 HWPROF B — Render CPI 4.34, 43.6% of cycles in instruction-fetch stalls; probe CPI 1.06 at 16 KB of hot code, 4.10 at 24 KB, 5.62 at 32 KB
11. 10-01 REL4 — 20 rendered + 20 replayed images/s, 1.3 ms per replay, but sky jitter, black tearing edges, giant fx triangles. 26.8 ms CPU per rendered frame. User: "we should go for a real 60!"
12. 10-01 n60-m1 (NRR milestone 1) — Hard freeze at race start. Cause: verify mode's state save/restore rolled back libctru's live GX queue (the emulator's GSP completes synchronously and cannot show it)
13. 10-02 n60-card2 — RUN V: 199,472 draws verified, bad=0, float24 max 0.011 px. RUN C: TT "silk 60 fps"; GP W median 14.2 ms = 57.3 fps outside log-flush windows (51.9 fps all windows), harsh hitch every ~1/5 lap
14. 10-02 n60-card3 — Race median 16.5 ms, 0 race hitches; save persisted across a power cycle. User: "omg it ran amazing!"
15. 10-03 interim legacy 3D — Black sky: the NRR sky consumed the section but could not draw under stereo
16. 10-03 n60-3d — No glitches; 3D race ~19.8 ms vs 16.5 ms 2D (frame-begin GPU wait 6.6 vs 3.3 ms) → 33 ms quantum, cost guard flapping, ~40 fps
17. 10-03 n60-3d2 — 2D 60 in GP and TT. 3D "40s and 50s": every race tripped the guard in the intro; steady 3D GP ≈ 48–57 fps
18. 10-07 n60-3d3 — Mute City 3D ~60 in end mode (hold 1 on 250–255 of 256 frames); Silence and Sand Ocean heavier (work 16–18 ms)
19. 10-08 n60-3d4 — Crash at boot on all three variants, no log: LCD framebuffers in VRAM, which the CIA maps read-only for the CPU
20. 10-08 card 4 (VRAM fix) — 3D GP courses 12–16: race windows hold 1 on 245–256 of 256 frames at work ≈15.5 ms → ≈59–60 fps. VERIFY variant bad=0. Heavy windows only on the post-race results table
21. 10-24 o3ds-3d ceiling probe (0c2668a) — Idling the GPU first did not make the frame cheaper (13.5 → 14.2 ms, 11.8 → 12.3 ms): Old 3DS 3D is CPU-bound; the deferred-wait plan was dropped. 1P 3D race with 16 machines: wall 18.6 ms = 53.7 fps. 3D multiplayer locked to 30 (ed04b17)
22. 10-25 v1.1 card 3 (3c0c7b7) — Race windows at a locked 60 (wall 16.71 ms, 3.8–4.1 ms of vblank slack); the steady governor still flapped 60/30 at D=1, fixed by the drift rule (a89289f)
23. 10-26 v1.1 card 5 (5e19320) — HOME-close hard lock (a 73 KB allocation in the 64DD save flush)
24. 10-26 v1.1 card 6 (ba0692e) — 1P 3D race 59.7–59.8 fps with one submission (baseline 56.4 / 59.3; two blocks per arm, so suggestive); 3D menu 40.2 → 44.6 fps; HOME close clean

The machine

Five phases

1 · Make it fit (09-24 → 09-29). RDRAM arena halved to 8 MiB, audio_table streamed; replay 49/49 with 2.65 MiB spare under simulation, and the console matched it byte for byte. One flag, -DGDX_3DS_O3DS=ON, makes the Old 3DS build.

2 · Find the costs (09-29 → 09-30). First race: 96 ms a frame, GPU busy 3.6 ms. The audio lock wait went 25.8 → 0.2 ms, ARMv6 SIMD made synthesis 2.2× cheaper, and frame skipping fixed the slow motion ("playable but nausea inducing"). A retained renderer won in the emulator and did almost nothing on hardware.

4 · The native race renderer (10-01 → 10-02). One small loop per race category, each sized to the 16 KB L1-I, writing command words from GPU-resident meshes. Game logic untouched and hash-checked every tick.

5 · 3D (10-03 → 10-08). The frame body is built once and resubmitted for the right eye with a per-eye uniform. The CPU fit, but frames missed vblank behind a 5–7 ms right-eye tail; interleaving per chunk and end-mode swaps gave Grand Prix windows at 59–60 on 10-08.

One submission for both eyes (v1.1.0)

After v1.0.0, a 3DS homebrew developer asked in a public comment whether the stereo path submits each eye separately, and suggested one command list for both. It did not re-encode the right eye, but every replayed list was its own GX queue entry, with its own GSP submit, completion interrupt and event-thread wake on core 0. Old 3DS probes put a clean citro3d draw at 10.9 µs and showed 0.4–0.5 ms of GPU dead time per backlogged queue entry (about 40 µs on New 3DS).

The fix uses the PICA200's second command-buffer channel: each stereo chunk ends by jumping to a short right-eye prologue, which jumps back into the same chunk, then to the finalize words. GSP gets one entry; the GPU runs the body twice and raises one interrupt. In the emulator every frame hash, game-state hash and screenshot matched the old path. On hardware, one build per console cycled four arms automatically:

The menu is the clean result (6 against 9 blocks, ±0.2 fps); the race line is suggestive. Dropping the mid-frame kick saved CPU but lost CPU/GPU overlap, so it netted zero. The New 3DS was neutral. Arm 3 ships as the v1.1.0 default on both consoles ([stereo] onesubmit = 1, onesubmit_narrow = 1).

Memory

The Old 3DS margin was thin at v1.0.0 and stayed the main source of post-release bugs:

• HOME close hard-locked the console (card 5). The 64DD save flush at exit wanted one contiguous 73 KB buffer; about 390 KB was free, but not in one piece. bad_alloc led to svcBreak, which locks instead of exiting. The flush now streams with no heap allocation, and a break during teardown exits. Card 6 logged a clean exit.
• The fast track path dropped after six course loads. A 96,768-byte allocation failed with 1.6 MB free: fragmentation, not exhaustion. The track now lives in one 777,360-byte arena reserved at boot (emulator-verified over a 6-course Grand Prix).
• About 320 KB moved off the main heap, mostly the bottom screen's art buffers, which had been static arrays in the program image.

What the Old 3DS forced

• Leaving the display list in races. A faster interpreter on a second core was enough on the New 3DS; at a third of the clock with no L2 and no spare core, the interpreter's own code did not fit in cache.
• A hardware profiler. The emulator models neither the I-cache nor the console's memory permissions, so its wins stopped predicting hardware. ARM11 performance counters became the judge.
• A memory diet. A smaller RDRAM arena, streamed audio tables, capped uploads, a pinned heap layout, and a high-water mark instead of a free-list figure that had been misread as headroom.
• Frame pacing as design. With a millisecond or two of margin, when the swap is requested matters: a swap gate, an end-of-frame mode, and a race-scoped frame-skip guard.

Still open

• Heavier venues (Silence, Sand Ocean) can step 3D down to 30.
• The results table after a race is still heavy.
• Brief audio gaps during loads. The vblank-rate crackle theory was measured and refuted.
• Memory remains a listed known issue.