PYTHON / NUMPY
Broadcasting rules
Predict the result shape of any element-wise NumPy operation, and use newaxis or keepdims to control how shapes line up.
What you will learn
- Right-align two shapes and pad with 1s to predict the result shape of a ufunc
- Use arr[:, None] to turn a (n,) array into a column that stretches across rows
- Keep reduction axes with keepdims=True so the result still aligns with the input
- Know that a length-1 axis is re-read with stride 0, never copied
Understanding Broadcasting rules
When a ufunc gets two arrays of different shapes, NumPy does not resize anything. It writes the two shapes down right-aligned, pads the shorter one with 1s on the left, and then compares them axis by axis. Each pair must either be equal or contain a 1; the output takes the larger of the two. So (2, 3) with (3,) becomes (2, 3) with (1, 3) and works, while (2, 3) with (2,) compares 3 against 2 and raises ValueError.
The mental model that keeps this straight is that a length-1 axis is not expanded in memory: NumPy sets that axis's stride to 0, so every step along it re-reads the same bytes. That is why adding a (1, 1000000) row to a (1000, 1000000) block costs no extra allocation for the row, and why a broadcast view from np.broadcast_to is read-only — many indices alias one element, so a write would be ambiguous. Broadcasting is a rule about how the iterator walks memory, not about building copies.
Because alignment happens from the right, a one-dimensional array of length n always behaves like a row, matching the last axis. If you want it to match rows instead, you have to give it a trailing 1 yourself: arr[:, np.newaxis] turns (n,) into (n, 1). The same reasoning explains keepdims=True on reductions: x.sum(axis=1) gives (n,) which aligns with columns, while x.sum(axis=1, keepdims=True) gives (n, 1) which aligns back with the rows it came from.
import numpy as np
table = np.array([[10, 20, 30],
[40, 50, 60]]) # shape (2, 3)
gain = np.array([[2],
[10]]) # shape (2, 1): stretched across columns
offset = np.array([1, 2, 3]) # shape (3,): stretched across rows
print(table.shape, gain.shape, offset.shape)
print(np.broadcast_shapes(table.shape, gain.shape, offset.shape))
print(table * gain)
print(table * gain + offset)Shapes are compared from the trailing axis backwards, each pair must be equal or 1, and a 1 is stretched by re-reading the same memory instead of copying it.
Worked examples
Mismatch versus an explicit new axis
Shows why (3,) plus (4,) is an error while (3, 1) plus (4,) produces a 3x4 grid.
import numpy as np
a = np.arange(3) # shape (3,)
b = np.arange(4) # shape (4,)
try:
a + b
except ValueError as e:
print("ValueError:", str(e).strip())
grid = a[:, np.newaxis] + b
print(a[:, np.newaxis].shape, b.shape, "->", grid.shape)
print(grid)Example explained
Line 1a + b compares the only axes, 3 against 4: neither is 1, so there is no way to align them.
Line 2a[:, np.newaxis] gives shape (3, 1), so the pairs become (3, 1) and (1, 4).
Line 3Both 1s stretch, giving (3, 4) where element [i, j] is a[i] + b[j].
Line 4This is the standard trick for turning two vectors into an outer-style table.
Broadcasting costs no memory
Inspects a broadcast view to show the stride-0 axis and why it is read-only.
import numpy as np
row = np.array([1.0, 2.0, 3.0])
big = np.broadcast_to(row, (4, 3))
print(big)
print(big.strides, row.strides)
print("writeable:", big.flags.writeable)
try:
big[0, 0] = 99.0
except ValueError as e:
print("write failed:", e)Example explained
Line 1big looks like 12 numbers but reuses the 3 floats stored in row.
Line 2The stride 0 on axis 0 means moving to the next row does not move the memory pointer at all.
Line 3Because all four rows alias the same bytes, NumPy marks the view read-only rather than letting one write change four "rows".
Line 4Arithmetic broadcasting uses exactly this stride-0 mechanism internally, which is why it adds no allocation.
keepdims keeps shapes aligned
Normalizing rows fails without keepdims because a (2,) array aligns with columns, not rows.
import numpy as np
x = np.array([[1., 2., 3.],
[4., 5., 6.]])
row_sums = x.sum(axis=1)
print(row_sums.shape, x.sum(axis=1, keepdims=True).shape)
try:
x / row_sums
except ValueError:
print("cannot broadcast (2, 3) with (2,)")
print(x / x.sum(axis=1, keepdims=True))Example explained
Line 1Summing over axis 1 removes that axis, leaving (2,) which right-aligns against the 3 columns.
Line 2keepdims=True leaves a length-1 axis in place, giving (2, 1) that aligns with the 2 rows.
Line 3The (2, 1) divisor stretches across columns, so each row is divided by its own sum.
Line 4Each printed row now sums to 1.0.
Important notes
Padding only happens on the left, never on the right, so (3, 1) and (3,) broadcast to (3, 3) rather than element-wise over 3 items.
Broadcasting is free for the inputs but not for the result: (10000, 1) with (1, 10000) allocates 100 million elements, so check the output shape before evaluating.
Common mistakes
Dividing by x.sum(axis=1) without keepdims: on a non-square array it raises ValueError, and on a square array it silently divides each column by a row sum, giving wrong numbers with no error.
Assuming a 1-D array is a column vector: adding a (n,) array to another (n,) array that was reshaped to (n, 1) quietly produces an (n, n) matrix instead of an (n,) result, which can exhaust memory for large n.
Expecting in-place operators to broadcast the output: a += b where a is (3,) and b is (2, 3) fails, because broadcasting may stretch inputs but can never grow the destination.
Try it yourself
Change, predict, then run
Build a 5x5 multiplication table from np.arange(1, 6) using np.newaxis, then subtract each row's mean using keepdims=True and print the row means of the result to confirm they are all 0.
Open the Python workspaceCheck your understanding
For a square array x with shape (3, 3), the expression x / x.sum(axis=1) runs without error. What does it actually compute?
- It divides column j of x by the sum of row j, so the rows are not normalized
- It divides each row by that row's own sum, identical to using keepdims=True
- It raises no error but returns the original array unchanged
- It divides every element by the total sum of all nine elements
Show answer
x.sum(axis=1) has shape (3,), which right-aligns with the last axis of x, so its entries are matched to columns even though they are row sums; the operation only works here because the array happens to be square. Option 1 is the tempting answer, but row normalization requires x.sum(axis=1, keepdims=True) with shape (3, 1) so the divisor aligns with the rows.