How Tensors Work
What is a Tensor?
From scalars to n-dimensional arrays
RANK
1
SHAPE
(4)
ELEMENTS
4
A vector is a 1D array of numbers. It has one axis. Think: a row of pixel intensities or word embedding.
Step 1 of 6: What is a Tensor?
From scalars to n-dimensional arrays
RANK
1
SHAPE
(4)
ELEMENTS
4
A vector is a 1D array of numbers. It has one axis. Think: a row of pixel intensities or word embedding.
Step 2 of 6: Tensor Shapes
Dimensions define the geometry of data
SHAPE
(2,3,4)
TOTAL
24
MEMORY
96B
A small tensor — fits easily in cache. Great for weights in a single layer.
Step 3 of 6: Reshaping
Same data, new perspective
OLD SHAPE
(2,3,4)
NEW SHAPE
(4,6,1)
VALID?
Yes
Reshape is valid: 4 x 6 x 1 = 24 elements — same data, new interpretation. No memory copy needed.
Step 4 of 6: Broadcasting
Automatic dimension alignment
A SHAPE
(3,4)
B SHAPE
(1,4)
RESULT
(3,4)
OK?
Yes
Broadcasting stretches size-1 dimensions to match. B's dim0 (1→3). No data is copied — NumPy/PyTorch use stride tricks.
Step 5 of 6: Element-wise vs Matrix Multiply
Two fundamental operation families
A SHAPE
(3,3)
B SHAPE
(3,3)
RESULT
(3,3)
Element-wise addition: A[i,j] + B[i,j]. Both tensors must have the same shape (or be broadcastable). 9 operations total.
Step 6 of 6: Tensors in Deep Learning
Batch × Channels × H × W
SHAPE
(4,3,8,8)
TOTAL
768
MEMORY
3.0KB
3 channels: typical for RGB input images. Batch size 4 means 4 images processed in parallel on the GPU.