[Review] The Data Model Determines Performance TODO
Concept
A data model sets the ceiling on performance before anything else, because it determines physical access paths, not just logical representation. Key design decides which lookups finish in a single seek and which turn into a full scan; the degree of normalization allocates cost between write-time duplication and read-time joins; and partitioning/clustering keys decide which range queries land on adjacent disk blocks. Indexes correct this structure after the fact, but they add write amplification and storage, so you can't pile on indexes indefinitely, and they can't fully rescue a fundamentally wrong model. So the practical order is: first write down which queries need to run at what frequency and latency target, then fit the model to that access pattern — doing it the other way around, fitting queries to a model chosen first, means paying for a migration later. The key point is that the multiplier you get from caching or more hardware is usually a constant factor, while a model change buys you an improvement in complexity order.
Most performance problems come from a schema mismatched to the access pattern, not from code that needs optimizing — and this is also the decision that's most expensive to reverse once data has accumulated.
Code & Formula
# [복습] 데이터 모델이 성능을 정한다 — 같은 데이터라도 접근 경로(키 설계)에 따라
# "단일 탐색"과 "전체 스캔"으로 갈리는 것을 두 가지 인덱스 구조로 비교한다.
orders = [
{"order_id": i, "user_id": i % 5, "amount": (i * 37) % 200}
for i in range(1, 21)
]
# 모델 A: order_id 로만 인덱싱 → "user_id=3의 주문 조회"는 전체 스캔이 필요
by_order_id = {o["order_id"]: o for o in orders}
def find_by_user_scan(user_id):
return [o for o in by_order_id.values() if o["user_id"] == user_id] # O(N)
# 모델 B: 접근 패턴("user_id로 자주 조회")에 맞춰 미리 파티셔닝/클러스터링
by_user_id = {}
for o in orders:
by_user_id.setdefault(o["user_id"], []).append(o) # 쓰기 시 중복 비용을 지불
def find_by_user_indexed(user_id):
return by_user_id.get(user_id, []) # O(1) 탐색 + 결과 크기만큼
target_user = 3
scan_result = find_by_user_scan(target_user)
indexed_result = find_by_user_indexed(target_user)
print("query: orders for user_id =", target_user)
print("model A (scan all orders):", [o["order_id"] for o in scan_result])
print("model B (pre-partitioned by user_id):", [o["order_id"] for o in indexed_result])
print("same result:", scan_result == indexed_result)
print("lesson: model B pays write-time cost to buy O(1) reads for the *actual* query pattern")
docs/code/algorithms/algorithms-81.py
Exercise
Pull the top three slowest queries in your current service, check their execution plans, and write down alternative key/partition designs that would turn each into a single index lookup, then compare the expected scanned-row counts.
Practical Connection
In Verex, 'all of one user's positions across every market' and 'all of one market's orders across every user' demand different access paths, so making both fast calls for separating storage structures by access pattern rather than trying to force one table design to cover both.
Where it lands in Jayverse
- Verex: name the query list and its latency target before any schema change. List positions-by-user, orders-by-market and resolution lookups with a target latency each, then verify with an execution plan that the design resolves to a single index lookup — make this the checklist an API review must pass.
- gitboard: track scanned-row counts for Verex's hot paths. A regression there is the early signal that an index patch has stopped buying enough, and that a genuine model change is due.
- Auditor: log which query pattern justified each table's key design. When storage is separated by access pattern, as Verex already does, record the reasoning so a future migration has it, not just the resulting schema.
Key expressions
| Expression | 뜻 · 쓰이는 자리 |
|---|---|
| set the ceiling on | ~의 상한선을 결정짓다 · 데이터 모델이 성능의 한계를 미리 정한다는 뜻. "A data model sets the ceiling on performance" |
| write amplification | 쓰기 증폭 · 인덱스를 추가할 때 실제 쓰기 작업이 늘어나는 현상. "they add write amplification and storage" |
| pile on | 계속 쌓아 올리다, 마구 추가하다 · 인덱스를 무한정 늘릴 수 없다는 뜻. "you can't pile on indexes indefinitely" |
| rescue | 잘못된 것을 구제하다, 살려내다 · 인덱스가 근본적으로 틀린 모델을 완전히 구하지는 못한다는 뜻. "they can't fully rescue a fundamentally wrong model" |
| mismatched to | ~와 맞지 않는, 어긋나는 · 스키마가 실제 접근 패턴과 안 맞을 때. "a schema mismatched to the access pattern" |
| expensive to reverse | 되돌리기에 비용이 많이 드는 · 데이터가 쌓인 뒤 모델을 바꾸기 어렵다는 뜻. "the most expensive to reverse once data has accumulated" |
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/dev-100-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.