PCA: the direction that keeps the most
Grey dots are centered data. Pink shows each point projected onto your direction. Blue and green are the principal components.
Your direction vs the best direction
Try thisTurn "your direction" slowly. The variance kept and the reconstruction error always add up to
the same total. PCA picks the direction where the kept part is largest, and so the error smallest.