Why this conversation
The strongest system-design answers start by rejecting the metric on the table and naming the one that matters. Doing it without sounding like a know-it-all is the skill: ask for the number, explain why it decides the user's experience, then propose the smallest change that would show it.
Dialogue
Lena: Average latency is 40 milliseconds and the error rate is under 0.1 percent. Users still complain that payment confirmations arrive late. Where would you start?
Jay: With a different number. The mean is fine; I would ask for the p99, and the p99 per region. A confirmation that arrives at 40 milliseconds ninety-nine times and at four seconds once is a service that feels slow, because the one is what people remember.
Lena: Suppose the p99 is 3.8 seconds. What is your hypothesis?
Jay: Two candidates, and I would check them in this order. First, a queue somewhere is serving in arrival order, so a burst of large payloads, say end-of-day statements, sits in front of the small confirmations. Second, a downstream call with a retry and a fixed timeout, where the timeout is the 3.8 seconds. Both show up as a flat shelf in the latency histogram rather than a smooth tail.
Lena: And if it is the queue?
Jay: Then the fix is not more capacity, it is ordering. Give small messages their own lane, or serve by remaining size so a confirmation overtakes a statement. That is the same idea as receiver-driven transports in datacenters: let the side that can see the congestion decide who goes next.
Lena: You are confident about the diagnosis before seeing the data.
Jay: Confident about the order of checks, not the answer. If the histogram has no shelf, I am wrong about both and I would look at GC pauses next. I would rather name my hypothesis and be corrected in ten minutes than be vague and right in a week.
Lena: What would you put on the dashboard tomorrow?
Jay: p99 next to the mean, split by message size, with the SLO drawn as a line. If the team cannot see the tail, they will keep optimising the average, and the average is already fine.
Key expressions
| Expression | 뜻 · 쓰이는 자리 |
|---|---|
| the mean is fine | 평균은 괜찮다(문제는 다른 곳에 있다) |
| the one is what people remember | 그 한 번이 사람들이 기억하는 것이다 |
| in this order | 이 순서로 |
| sits in front of | ~의 앞에 앉아 있다(막고 있다) |
| a flat shelf in the histogram | 히스토그램의 평평한 턱(특정 값에 몰린 구간) |
| its own lane | 자기 차선(별도 처리 경로) |
| overtakes | 추월한다 |
| confident about the order of checks, not the answer | 답이 아니라 확인 순서에 확신이 있다 |
| I would rather … than … | …보다 …하는 쪽이 낫다 |
| drawn as a line | 선으로 그려진(기준선으로 표시된) |
| keep optimising the average | 계속 평균을 최적화하다 |