JIT Tiering, Warmup, and Deoptimization (Deopt) TODO
Concept
Modern VMs don't optimize all code from the start — they use a tiered strategy. Code first runs on an interpreter or a fast-emitting baseline compiler while the VM collects call counts, loop-iteration counts, and type profiles; only hot code that crosses a threshold gets recompiled by the optimizing compiler. The optimizing tier uses the profile to make assumptions such as "this argument is always an integer" or "this call target is always the same function," inlining and specializing on those assumptions, and it embeds guards in the code to check them. When a guard fails, deoptimization (deopt) kicks in, rolling the optimized frame's state back into an interpreter frame and continuing execution on the slow path. This is why programs have a low-performance warmup period at the start, and why, if assumptions keep breaking, repeated recompilation and deopt can make performance fall off a cliff.
A microbenchmark that ignores warmup produces numbers far slower or faster than reality and leads to the wrong optimization decisions. It's also common in practice for a single polymorphic object shape used at one call site to push a hot loop into deopt, dropping throughput by several times.
Code & Formula
# JIT 계층화·워밍업·역최적화(deopt) — 호출 횟수로 티어를 올리고, 타입 가정이 깨지면 deopt한다.
class JitFunction:
def __init__(self, threshold=5):
self.call_count = 0
self.threshold = threshold
self.tier = "interpreter"
self.assumed_type = None
self.deopt_count = 0
def call(self, x):
self.call_count += 1
if self.tier == "interpreter" and self.call_count >= self.threshold:
self.tier = "optimized"
self.assumed_type = type(x) # 관측한 타입으로 특수화(가정 수립)
if self.tier == "optimized":
if type(x) is not self.assumed_type: # guard 실패
self.tier = "interpreter" # deopt: 인터프리터 프레임으로 복귀
self.deopt_count += 1
self.assumed_type = None
self.call_count = 0
else:
return x * 2 # 특수화된 빠른 경로
return x * 2 # 일반(느린) 경로
fn = JitFunction(threshold=3)
trace = []
for x in [1, 2, 3, 4, 5, "oops", 6, 7, 8, 9]:
tier_before = fn.tier
result = fn.call(x)
trace.append((x, tier_before, fn.tier, result))
for x, before, after, result in trace:
marker = " <- DEOPT" if before == "optimized" and after == "interpreter" else ""
print(f"call({x!r:>7}): tier {before:>11} -> {after:<11} result={result}{marker}")
print(f"\n총 deopt 횟수: {fn.deopt_count}")
docs/code/algorithms/algorithms-24.py
Exercise
In Node.js, benchmark a hot function called with a single type only, versus calling it with a mix of differently-shaped objects partway through, and check the recompilation/deopt logs with --trace-deopt and --trace-opt.
Practical Connection
A hot loop written in TypeScript, like the Verex matching engine's, needs to keep order-object shapes consistent to stay in the optimized tier, and when benchmarking, the warmup period needs to be discarded for the p99 numbers to mean anything.
Where it lands in Jayverse
- Verex: audit the matching engine's hot-path order object for shape consistency as a standing check. Same fields, same types, same order across call sites — not a one-time fix, since a single polymorphic call site can silently push a hot loop out of the optimized tier and cut throughput for no visible reason.
- Verex: add a warmup-discard step to CI's perf checks. The p99 numbers feeding the performance budget (see algorithms-51) are only meaningful once the JIT's warmup period is excluded, so make that exclusion part of the benchmark script, not a manual step someone forgets.
Key expressions
| Expression | 뜻 · 쓰이는 자리 |
|---|---|
| warmup | 초반 저성능 구간, 예열 단계 · "a low-performance warmup period at the start" |
| fall off a cliff | 성능이 급격히 곤두박질치다 · "performance fall off a cliff" |
| guard (code) | 가정이 맞는지 검사하는 코드 조각 · "it embeds guards in the code to check them" |
| deoptimization | 최적화를 해제하고 느린 경로로 되돌리는 것 · "deoptimization (deopt) kicks in" |
| polymorphic | 여러 형태를 가지는, 다형적인 · "a single polymorphic object shape used at one call site" |
| hot (code) | 자주 실행되어 최적화 대상이 되는 · "only hot code that crosses a threshold" |
| discard | 버리다, 계산에서 제외하다 · "the warmup period needs to be discarded" |
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/dev-100-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.