Differentiation, Gradients, and the Chain Rule TODO
Concept
The derivative is the coefficient of the linear function that best approximates a function near a point, so its value tells you how many times the output changes for a small change in the input. For a multivariable function, the gradient collects the partial derivatives with respect to each variable into a vector; it points in the direction of steepest increase at that point, and its magnitude is that rate of increase. The chain rule states that the derivative of a composite function is the product of the derivatives at each stage, and in the multivariable case this generalizes to a product of Jacobian matrices. Backpropagation computes this Jacobian product from the output side toward the input side, reusing intermediate results so that gradients can be obtained far more cheaply when there are many parameters. Gradient descent repeatedly moves a small step in the direction opposite the gradient to find a local minimum; the practical crux is that too large a step size (learning rate) causes divergence, while too small a one makes convergence painfully slow.
Optimization, curve fitting, and parameter tuning all run on gradients, and derivatives are exactly the language you need whenever you have to reason about a function's sensitivity — as with market-maker pricing curves. Without the chain rule you cannot compute the sensitivity of a composed system.
Code & Formula
# Day 27 — 미분·기울기·연쇄법칙
# 수치미분으로 도함수를 근사하고, 연쇄법칙 f(g(x))의 도함수를 직접 계산과 비교한다.
def numerical_diff(f, x, h=1e-6):
return (f(x + h) - f(x - h)) / (2 * h)
def f(x):
return x ** 3 + 2 * x
def f_prime_exact(x):
return 3 * x ** 2 + 2
x0 = 2.0
print(f"f'({x0}) 수치미분 근사 = {numerical_diff(f, x0):.6f}")
print(f"f'({x0}) 해석적 값 = {f_prime_exact(x0):.6f}")
# 연쇄법칙: h(x) = g(f(x)), g(u) = sin(u) 라 하면 h'(x) = g'(f(x)) * f'(x)
import math
def g(u):
return math.sin(u)
def h(x):
return g(f(x))
def h_prime_chain_rule(x):
g_prime = math.cos(f(x)) # g'(u) = cos(u), u = f(x)
return g_prime * f_prime_exact(x)
print(f"\nh'({x0}) 수치미분 근사 = {numerical_diff(h, x0):.6f}")
print(f"h'({x0}) 연쇄법칙 계산 = {h_prime_chain_rule(x0):.6f}")
Exercise
Pick a simple multivariable function, compute its gradient by hand, compare it against a numerical derivative, then run a few steps of gradient descent yourself and observe the learning-rate threshold where convergence turns into divergence.
Practical Connection
An LMSR market maker's price is defined as the partial derivative of the cost function, so derivatives and gradients feed directly into understanding Verex's price calculation and its sensitivity to slippage.
Where it lands in Jayverse
- Verex: bound slippage with the LMSR gradient/Hessian. Beyond price being the cost function's derivative, use its gradient (or Hessian, for curvature) to add a test that asserts price sensitivity stays inside a configured band as the liquidity parameter b changes.
- OFA: document the solver's learning-rate schedule. Since gradient-descent-style search underlies bidding/pricing optimization, pick and write down a learning-rate schedule for the ATLAS-style intent solver, using this exercise's divergence-vs-slow-convergence threshold as the design check.
- Number: standardize one gradient-descent utility. Build a small shared utility (with the learning-rate diagnostic from the exercise) that every Number research or backtest script reuses, instead of hand-rolling gradient descent per script.
Key expressions
| Expression | 뜻 · 쓰이는 자리 |
|---|---|
| best approximates | 가장 근접하게 근사하다 · 도함수가 함수를 가장 잘 근사하는 선형 함수임을 설명할 때. "the linear function that best approximates a function" |
| steepest increase | 가장 가파른 증가(방향) · 그래디언트가 가리키는 방향을 설명하는 수학 용어. "the direction of steepest increase at that point" |
| generalizes to | ~로 일반화되다 · 단변수 개념이 다변수·행렬 형태로 확장될 때. "this generalizes to a product of Jacobian matrices" |
| crux | 핵심(가장 중요한 지점) · 실질적으로 중요한 문제의 본질을 가리키는 단어. "the practical crux is that too large" |
| painfully slow | 답답할 정도로 느린 · 학습률이 너무 작을 때의 수렴 속도를 표현. "makes convergence painfully slow" |
| reuse intermediate results | 중간 계산 결과를 재사용하다 · 역전파가 계산 비용을 줄이는 방식을 설명. "reusing intermediate results so that gradients" |
| sensitivity to slippage | 슬리피지(가격 변동)에 대한 민감도 · 가격 곡선이 입력 변화에 얼마나 민감한지 가리킴. "its sensitivity to slippage" |
| LMSR | 로그 마켓 스코어링 규칙(Logarithmic Market Scoring Rule) · Verex 가격이 이 비용함수의 편미분으로 정의됨. "An LMSR market maker's price is defined as" |
| Jacobian matrix | 야코비 행렬 · 다변수 함수의 모든 편미분을 모은 행렬, 체인룰의 다변수 확장에 등장. "generalizes to a product of Jacobian matrices" |
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/math-50-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.