Dot Product, Norm, Cosine Similarity TODO
Concept
The dot product of two vectors is defined as the sum of the products of corresponding components, and geometrically it equals the product of the two vectors' magnitudes times the cosine of the angle between them. A norm is a function measuring vector magnitude: the L2 norm is the square root of the dot product of a vector with itself, L1 is the sum of the absolute values of the components, and L∞ is the maximum absolute value. The Cauchy-Schwarz inequality says the absolute value of the dot product never exceeds the product of the two norms, which is exactly why dividing the dot product by the two norms always gives a value between -1 and 1, making cosine similarity well-defined. Cosine similarity ignores vector magnitude and compares direction only, so it's used instead of Euclidean distance when you want to exclude differences in document length or scale. A dot product of zero means orthogonality, which is the starting point for concepts like projection and least squares.
Practical problems like embedding search, similarity ranking, and how results change depending on whether you normalize, ultimately come down to which norm you're measuring with and whether you normalized magnitude.
Code & Formula
# 내적·노름·코사인 유사도 — L1/L2/L∞ 노름, 코사인 유사도, 코시-슈바르츠 부등식을 확인한다.
import numpy as np
u = np.array([3.0, 4.0, 0.0])
v = np.array([1.0, 2.0, 2.0])
dot = np.dot(u, v)
print(f"u={u}, v={v}")
print(f"내적 u·v = {dot}")
l1 = np.linalg.norm(u, 1)
l2 = np.linalg.norm(u, 2)
linf = np.linalg.norm(u, np.inf)
print(f"\nu의 노름: L1={l1}, L2={l2}, L∞={linf}")
cos_sim = dot / (np.linalg.norm(u) * np.linalg.norm(v))
print(f"\n코사인 유사도 cos(u,v) = {cos_sim:.4f}")
# 코시-슈바르츠: |u·v| <= ||u|| * ||v|| → 이 부등식 덕분에 cos_sim이 항상 [-1, 1] 안에 있다
cauchy_schwarz_bound = np.linalg.norm(u) * np.linalg.norm(v)
print(f"|u·v| = {abs(dot):.4f} <= ||u||*||v|| = {cauchy_schwarz_bound:.4f} ? {abs(dot) <= cauchy_schwarz_bound}")
# 크기가 다른 두 벡터라도 방향이 같으면 코사인 유사도는 1
w = u * 10 # u와 방향은 같고 크기만 10배
print(f"\nw = 10*u 의 코사인 유사도(u,w) = {np.dot(u, w) / (np.linalg.norm(u) * np.linalg.norm(w)):.4f} → 스케일 무시, 방향만 비교")
Exercise
For the same dataset, implement Euclidean-distance nearest neighbors and cosine-similarity nearest neighbors separately, and check how the results change before and after normalizing vectors to unit norm.
Practical Connection
This is used directly when clustering similar wallets by on-chain address behavior vectors or wiring up log/document search, and more generally it underlies predictive models like least squares and regression.
Where it lands in Jayverse
- Number: make normalization an explicit config, not a hidden default. If Number ships a "similar reading" feature, decide and document cosine similarity versus Euclidean distance and whether vectors are unit-normalized, since the two give different neighbors on the same data.
- Personas: cluster wallets by direction, not magnitude. Grouping wallets by on-chain behavior vectors for persona assignment should use cosine similarity so a whale and a casual wallet with the same behavior pattern land in the same cluster instead of being split by transaction-count scale.
- gitboard: use cosine similarity for "find similar past incidents." A log-search feature over gitboard's incident history is exactly the embedding-search case this card describes; pick the norm deliberately rather than defaulting to whatever the library ships.
Key expressions
| Expression | 뜻 · 쓰이는 자리 |
|---|---|
| well-defined | (수학적으로) 명확히 정의된 · 코사인 유사도 값이 항상 -1~1 사이로 보장될 때 · "making cosine similarity well-defined" |
| come down to | 결국 ~로 귀결되다 · 실제 문제들이 결국 어떤 노름을 쓰는지로 좁혀질 때 · "ultimately come down to which norm you're measuring with" |
| starting point for | ~의 출발점 · 내적이 0이면 직교라는 사실이 다른 개념으로 이어질 때 · "the starting point for concepts like projection and least squares" |
| orthogonality | 직교성(벡터 간 내적이 0인 상태) · 두 벡터가 전혀 상관없는 방향일 때 · "A dot product of zero means orthogonality" |
| unit norm | 단위 노름(크기를 1로 정규화한 상태) · 벡터를 정규화할 때 기준이 되는 크기 · "normalizing vectors to unit norm" |
| ignores vector magnitude | 벡터의 크기(스케일)를 무시하다 · 코사인 유사도가 방향만 비교하는 이유를 설명할 때 · "Cosine similarity ignores vector magnitude" |
| Cauchy-Schwarz inequality | 코시-슈바르츠 부등식 · 내적이 두 노름의 곱을 넘지 않음을 보장하는 정리 · "The Cauchy-Schwarz inequality says the absolute value" |
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/math-50-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.