NumPy essentials

Lesson 42 of 62, about 5 min

NumPy is the foundation of numerical Python. Its core object is the n-dimensional array, which stores numbers compactly and lets you do maths on whole arrays at once, without writing loops. pandas, scikit-learn and every deep learning library are built on it.

Creating arrays

python

import numpy as np

a = np.array([1, 2, 3, 4])
m = np.array([[1, 2, 3], [4, 5, 6]])
print(a)
print(m)
print(a.shape, m.shape)
print(a.dtype, m.ndim)

shape is a tuple of sizes along each axis. A 2 by 3 matrix has shape (2, 3).

Handy constructors:

python

import numpy as np

print(np.zeros(3))
print(np.ones((2, 2)))
print(np.arange(0, 10, 2))
print(np.linspace(0, 1, 5))
print(np.eye(3))
np.random.seed(0)
print(np.random.rand(2, 2).round(2))

Vectorised maths

Operations apply element by element, and they run in optimised C code, so they are far faster than Python loops.

python

import numpy as np

a = np.array([1, 2, 3, 4])
b = np.array([10, 20, 30, 40])
print(a + b)
print(a * b)
print(a ** 2)
print(np.sqrt(b))
print(a > 2)

Broadcasting

When shapes differ, NumPy stretches the smaller array to fit, as long as the shapes are compatible. A scalar is broadcast to every element; a row is broadcast down every row.

python

import numpy as np

m = np.array([[1, 2, 3], [4, 5, 6]])
print(m * 10)
print(m + np.array([100, 200, 300]))
print(m - m.mean(axis=0))      # centre each column

Indexing and slicing

python

import numpy as np

m = np.arange(12).reshape(3, 4)
print(m)
print(m[1, 2])       # row 1, column 2
print(m[0])          # first row
print(m[:, 1])       # second column
print(m[1:, 2:])     # sub matrix

Boolean masks

Select elements that satisfy a condition. This pattern is everywhere in data work.

python

import numpy as np

marks = np.array([45, 82, 67, 91, 30, 75])
print(marks[marks >= 60])
print((marks >= 60).sum(), "students passed")
marks[marks < 33] = 33          # floor the failing marks
print(marks)

Aggregations

python

import numpy as np

m = np.array([[1, 2, 3], [4, 5, 6]])
print(m.sum(), m.mean(), m.max(), m.std().round(3))
print(m.sum(axis=0))    # down the columns
print(m.sum(axis=1))    # across the rows
print(m.argmax())       # index of the largest value in the flattened array

axis=0 collapses rows (result per column); axis=1 collapses columns (result per row).

Reshaping

python

import numpy as np

a = np.arange(6)
print(a.reshape(2, 3))
print(a.reshape(3, -1))    # -1 means "work it out"
print(a.reshape(2, 3).T)   # transpose
print(a.reshape(2, 3).flatten())

Linear algebra

Matrix multiplication uses @. It is the operation at the heart of linear regression and neural networks.

python

import numpy as np

X = np.array([[1, 2], [3, 4]])
w = np.array([0.5, -1])
print(X @ w)
print(X @ X)
print(np.linalg.inv(X).round(2))
print(np.dot(w, w))

Why it matters for machine learning

A dataset is a 2D array: one row per sample, one column per feature. A model's parameters are arrays. Training is repeated array arithmetic. Getting comfortable with shapes, axes and broadcasting will make every later lesson easier to read.

python

import numpy as np

X = np.array([[450], [600], [750], [900]], dtype=float)
y = np.array([9000, 12000, 15500, 18000], dtype=float)
w = 20.0
b = 0.0
predictions = X[:, 0] * w + b
print(predictions)
print("mean squared error:", ((predictions - y) ** 2).mean())

Practice

Create a 4 by 4 array of the numbers 1 to 16 and print its diagonal, its second column and the sum of each row.
Generate 100 random numbers from a normal distribution and count how many are more than one standard deviation from the mean.
Standardise a 2D array so every column has mean 0 and standard deviation 1, using broadcasting.