PCA and SVD TODO
Concept
SVD decomposes an arbitrary real matrix A as A = UΣVᵀ, where U and V are orthogonal matrices and Σ is a diagonal matrix holding the nonnegative singular values in decreasing order. Geometrically, this means any linear transformation can be viewed as the composition of 'rotation (or reflection) → axis-wise scaling → rotation'; the size of the singular values indicates how much each direction is stretched, and the number of singular values close to zero indicates how rank-deficient the matrix is. The truncated SVD keeping only the top k singular values has the property (Eckart–Young) of being the optimal rank-k approximation under the Frobenius norm, which is the theoretical basis for dimensionality reduction and noise removal. PCA is the procedure of centering the data by column means and then finding the eigenvectors of the covariance matrix — in the SVD of the centered data matrix, the columns of V are exactly the principal components, and the squared singular values are proportional to the variance explained by each component. In other words, PCA is the statistical interpretation of SVD, and forgetting to center the data is the most common practical pitfall that makes the two results diverge.
Compressing high-dimensional metrics or extracting the dominant axes of variation from correlated signals comes up repeatedly in anomaly detection, risk decomposition, and feature extraction, and reading condition number and rank deficiency is also a basic tool for diagnosing numerical instability.
Code & Formula
# PCA·SVD — 상관된 2D 합성 데이터에서 주성분(최대 분산 방향)을 SVD로 직접 구한다.
import numpy as np
rng = np.random.default_rng(0)
n = 300
# x, y가 강하게 상관되도록 만든 2D 데이터 (주된 퍼짐 방향이 대략 45도가 되게)
t = rng.normal(0, 3, n)
x = t + rng.normal(0, 0.3, n)
y = t * 0.6 + rng.normal(0, 0.3, n)
X = np.column_stack([x, y]) # shape (n, 2)
X_centered = X - X.mean(axis=0) # PCA는 평균을 원점으로 옮긴 뒤 분산 방향을 찾는다
U, S, Vt = np.linalg.svd(X_centered, full_matrices=False)
print(f"특이값(Σ): {S}")
print("주성분 방향(V의 행, 분산이 큰 순서):\n", Vt)
pc1 = Vt[0]
angle_deg = np.degrees(np.arctan2(pc1[1], pc1[0]))
print(f"\n제1주성분(PC1) 방향 벡터 = {pc1}, x축 대비 각도 ≈ {angle_deg:.1f}도")
# 데이터를 PC1 축 하나에 투영 (2D -> 1D 차원축소)
projected = X_centered @ pc1
explained_var_ratio = (S ** 2) / np.sum(S ** 2)
print(f"\nPC1 하나로 설명되는 분산 비율 = {explained_var_ratio[0]:.4f}")
print(f"투영된 1D 값 예시(앞 5개): {np.round(projected[:5], 3)}")
Exercise
Generate synthetic 2-3 dimensional data with strong correlation, center it and compute the SVD, derive the variance explained by each principal component from the squared-singular-value ratios, and compare how the results differ if you skip centering.
Practical Connection
Arranging multiple markets' price time series as a matrix and extracting principal components can separate a common factor like 'overall market direction' from movements unique to individual markets, which can be used for anomalous price detection and gauging risk concentration.
Where it lands in Jayverse
- Number: publish the PCA-on-price-matrix routine as a licensed reading. Arrange multiple markets' price series as a matrix, extract the top components, and publish the loadings as a Number indicator under the tokenized-index shape (data, licence, expiry in one token) already used elsewhere on Number.
- Verex: use variance-explained ratio as a market-health check. A single component with unexpectedly high loading where markets should be independent is worth flagging for review — correlated or manipulated-looking price movement, not just noise.
- Auditor: require centering to be logged explicitly. Forgetting to center the data before SVD is the most common pitfall; any PCA-based check should record that step so an uncentered result isn't mistaken for a real signal.
Key expressions
| Expression | 뜻 · 쓰이는 자리 |
|---|---|
| rank-deficient | 랭크(계수)가 부족한 · 행렬의 결함 정도를 말할 때. "how rank-deficient the matrix is" |
| diverge | (결과가) 갈라지다, 달라지다 · 두 계산 결과가 서로 어긋날 때. "makes the two results diverge" |
| the theoretical basis for | ~의 이론적 근거 · 어떤 방법이 왜 타당한지 설명할 때. "the theoretical basis for dimensionality reduction" |
| gauge (verb) | 가늠하다, 측정해 판단하다 · 리스크 집중도를 파악할 때. "gauging risk concentration" |
| diagnose | (문제·상태를) 진단하다 · 수치적 불안정성을 찾아낼 때. "diagnosing numerical instability" |
| dominant axes of variation | 변동의 지배적인 축(가장 큰 변화 방향) · 데이터에서 주요 패턴을 뽑아낼 때. "extract the dominant axes of variation" |
| pitfall | 흔히 빠지는 함정 · 계산 시 자주 저지르는 실수를 경고할 때. "the most common practical pitfall" |
| SVD | 특이값분해(Singular Value Decomposition) · 임의의 실수 행렬을 회전·스케일링·회전으로 분해하는 도구. "SVD decomposes an arbitrary real matrix A" |
| PCA | 주성분분석(Principal Component Analysis) · 데이터를 열 평균으로 중심화한 뒤 공분산 행렬의 고유벡터를 구하는 절차. "the procedure of centering the data by column means" |
| Eckart–Young | 에카트–영 정리 · 절단된 SVD가 프로베니우스 노름 기준 최적의 저랭크 근사임을 보장하는 정리. "has the property (Eckart–Young) of being the optimal rank-k approximation" |
| Frobenius norm | 프로베니우스 노름 · 절단 SVD의 최적성을 판단하는 기준이 되는 행렬 노름. "the optimal rank-k approximation under the Frobenius norm" |
| condition number | 조건수 · 수치적 불안정성을 진단하는 기본 지표. "reading condition number and rank deficiency" |
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/math-50-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.