Linear Algebra Matrix

Video Resources#

#linear-algebra #vectors

Video: Vectors | Chapter 1, Essence of linear algebra# Video: Linear combinations, span, and basis vectors | Chapter 2, Essence of linear algebra#

  • What to Learn:
    • Definition of vectors (geometrical and algebraic perspectives)
    • Vector operations: addition, scalar multiplication, dot product, and cross product
    • Vector norms (magnitude/length): L1, L2 norms
    • Basis vectors and the concept of a vector space
    • Linear combinations and span of vectors
  • Why It's Important:
    • In machine learning, data points are often represented as vectors.
    • The dot product and norms are used in distance-based models (e.g., k-NN, SVMs, etc.).

Geometrical Perspective#

A vector is a line segment with both magnitude and direction.

We can think of a vector as an arrow where it's tail sits at the origin of a plane, and points to either 0 or any give scalar.

Characteristics#

  • Usually represented by an arrow
  • The length of the arrow represents the magnitude
  • The orientation of the arrow represents the direction
  • Commonly denoted by letters with a tilde above v~ \tilde{v} ![[Vector.png]]

Examples#

  • In 2D space a vector can represent a displacement from point A to point B
  • Moving 3 units East and 1 unit North as the example above shows

Algebraic Perspective#

A vector is an order list of numbers that describes its components in various axes in space.

2D Vector simply means there's two numbers

[31]v=(v3,v1)\begin{bmatrix} {3} \\ {1} \\ \end{bmatrix} \mathbf{v =} \begin{pmatrix} v_{3}, v_{1} \end{pmatrix}

Similarly 3D Vector means there's three numbers

[312]v=(v3,v1,v2)\begin{bmatrix} {3} \\ {1} \\ {2} \end{bmatrix} \mathbf{v =} \begin{pmatrix} v_{3}, v_{1}, v_{2} \end{pmatrix}

Characteristics#

  • Each component represents how much a vector moves along an axis.
  • Vectors in higher dimensions are generalizations
(Rn),v=(v1,v2,…,vn)(\mathbb{R}^n), \mathbf{v =} \begin{pmatrix} v_{1}, v_{2}, \dots, v_{n} \end{pmatrix}

Vector Notation#

v=(v1,v2,…,vn)v=[312] \mathbf{v =} \begin{pmatrix} v_{1}, v_{2}, \dots, v_{n} \end{pmatrix} \mathbf{v =} \begin{bmatrix} {3} \\ {1} \\ {2} \end{bmatrix}

Vector Operations#

Addition#

The sum of two vectors is a vector obtained from adding their corresponding components

  • Geometrical - Place the tail of the second vector at the head of the first vector. The sum vector points from the tail of the firs vector to the head of the second vector ![[Pasted image 20250106214057.png]]

  • Algebraic Formula

u+v=(ux​+vx​,uy​+vy​,uz​+vz​)u+v=(ux​+vx​,uy​+vy​,uz​+vz​) \\ u=(1,2,3)v=(4,5,6) \mathbf{u} = (1, 2, 3) \\ \mathbf{v} = (4, 5, 6) u+v=(1+4,2+5,3+6)=(5,7,9)u+v= (1 + 4, 2 + 5, 3+6) = (5, 7, 9)

Scalar Multiplication#

Multiplying a vector by a scalar (number) scales the magnitude of the vector in its direction, multiplying by a negative number would scale to the opposite direction.

  • Algebraic Formula
c⋅v=(c⋅vx,c⋅vy,c⋅vz)c \cdot v = (c \cdot v_{x}, c \cdot v_{y}, c \cdot v_{z})

Where c is the scalar and v=(vx,vy,vz)\text{Where } c \text{ is the scalar and } v = (v_{x}, v_{y}, v_{z})

Dot Product#

Video Resource: Dot products and duality | Chapter 9, Essence of linear algebra#

Cross Product#

Video Resource:

Cross products | Chapter 10, Essence of linear algebra#