explainer
Tensors: read shapes, select values, and move axes
Learn tensors through a robot image batch. Explore shape, indexing, slicing, and axis order, then compare reshape with transpose using a runnable Python example.
What you will learn
- Read an array's shape and name the meaning of each axis.
- Locate a value and predict the shape of a basic slice.
- Distinguish tensor order from matrix rank.
- Explain why reshaping an array differs from transposing its axes.
Before you start
- Vectors and coordinates
- Multiplication and counting from zero
A robot image batch can have four axes while describing a scene on a flat camera sensor. Those axes might mean image, row, column, and color channel. Naming them makes the shape useful.
This lesson follows one value through indexing, slicing, and an axis change. A small batch lets you check every result without a machine learning library.
Start with an array of values
In machine learning software, a tensor usually means a multidimensional array with a common element type. The TensorFlow tensor guide uses this array-based definition. We will work with dense, rectangular arrays of numbers.
An axis is one position in the indexing scheme. An image can use one axis for rows, one for columns, and one for channels. An axis need not represent a direction in physical space: the batch axis selects an example.
The word has a deeper mathematical meaning too. In physics, a tensor describes an object whose components follow specific transformation rules when coordinates or bases change. NPTEL's tensor transformation notes develop that definition.
This lesson covers the software usage. Putting measurements into a four-axis array alone does not establish a physical tensor transformation law.
Separate shape, order, and size
The shape lists the length of each axis. The order, also called ndim or tensor rank in some libraries, counts the axes. The size counts all elements by multiplying the axis lengths.
| Example | Shape | Order | Elements |
|---|---|---|---|
| One loss value | () | 0 | 1 |
| Three features | (3,) | 1 | 3 |
| A small grayscale image | (2, 3) | 2 | 6 |
| One RGB image | (2, 3, 3) | 3 | 18 |
| Two RGB images | (2, 2, 3, 3) | 4 | 36 |
The comma in (3,) marks a one-item Python tuple. A scalar has no axes and still contains one value. An axis of length zero produces an empty array, such as shape (0, 3) with zero elements.
Explore an image batch
Our batch contains two images. Each image has two rows, three columns, and three channels: red, green, and blue. The values run from 0 to 35 so you can trace them; they are invented data.
The initial layout is NHWC: number of images, height, width, channels. Its shape is (2, 2, 3, 3). The explorer holds the image and channel fixed to show a two-dimensional slice.
Start with the selected value 16. It belongs to image 0, row 1, column 2, green channel 1. Choose NCHW (channels first) and watch its index change while the value stays 16.
Try Next element in each layout. It advances through the current logical flat order, where the last index changes fastest. Reset example restores the starting selection and layout.
Select a value or keep a slice
Indices count from zero. For a length-three axis, the valid nonnegative indices are 0, 1, and 2. To select one scalar from our four-axis batch, supply four indices:
X[0, 1, 2, 1] = 16
With NumPy-style basic indexing, an integer selects one position and removes that axis from the result. A colon keeps the full axis. A range such as 0:1 keeps an axis of length one; its stop is exclusive. NumPy's indexing guide specifies these rules.
For the NHWC batch:
X[0]selects one image, leaving shape(2, 3, 3).X[0:1]keeps a one-image batch, leaving shape(1, 2, 3, 3).X[0, :, :, 1]selects the green channel of image 0, leaving shape(2, 3).X[0, 1, 2, :]selects all three channels at one pixel, leaving shape(3,)with values[15, 16, 17].
A one-image batch and an individual image can hold the same values while exposing different shapes. A model that expects a batch axis still needs that axis when the batch contains one example.
Trace one value through two layouts
The element count follows directly from the four axis lengths:
2 × 2 × 3 × 3 = 36 elements
In row-major NHWC order, channel changes fastest. After all channels, advance the column; after all columns, advance the row. With batch b, row r, column c, and channel k, the zero-based flat offset is:
offset = ((b × H + r) × W + c) × C + k
Substitute b = 0, r = 1, c = 2, k = 1, H = 2, W = 3, and C = 3. The result is ((0 × 2 + 1) × 3 + 2) × 3 + 1 = 16. Offset 16 means the seventeenth element when counting from one.
Moving the channel axis to position 1 gives NCHW, with shape (2, 3, 2, 3). The same location becomes Y[0, 1, 1, 2] = 16. Its flat offset in the new logical order is ((0 × 3 + 1) × 2 + 1) × 3 + 2 = 11.
Value and offset matched initially because we filled the original batch with consecutive numbers. The axis change separates them. The relationship is Y[b, k, r, c] = X[b, r, c, k] for every location.
Compare reshape and transpose
Reshape changes how an ordered sequence of elements fits into a shape. It must preserve the element count. For row-major order, flatten the rows and regroup that sequence into the new dimensions. NumPy's reshape reference defines this ordering explicitly.
Transpose permutes axes. For a matrix, it exchanges rows and columns, as the matrix transpose and inverse lesson demonstrates. For more axes, an explicit permutation states which original axis supplies each new position. The NHWC-to-NCHW permutation is (0, 3, 1, 2), as defined by NumPy's transpose reference.
Compare these results from one matrix:
Original (2, 3): [[0, 1, 2], [3, 4, 5]]
Reshape to (3, 2): [[0, 1], [2, 3], [4, 5]]
Transpose to (3, 2): [[0, 3], [1, 4], [2, 5]]
Both results have six elements and shape (3, 2). Their entries differ. Changing a shape to match a model's expected dimensions can silently assign values to the wrong channels or locations.
Matrix multiplication gives shape a different role: a (m, n) matrix can multiply a (n, p) matrix to produce (m, p). Each output entry sums products across the shared dimension; reshaping alone does not perform that calculation.
Check meaning and storage
Shape does not record units, channel names, or feature order. Two vectors can both have shape (3,) while listing different features. Before taking their dot product, align the meanings of matching entries.
pandas adds explicit row and column labels to tabular data. Its lesson shows how label-based selection and alignment differ from selecting an array by position.
An array's dtype specifies how it stores each value. A dense batch of 32 RGB images at 224 × 224 has 4,816,896 elements. Using four-byte float32 values requires 19,267,584 bytes for the element buffer, excluding framework overhead and other model data.
The explorer reports logical offsets. Real arrays can use strides, which specify how far to move in storage along an axis. A transpose can share existing storage through changed strides; a reshape may require a copy. Shape alone cannot tell you whether an operation copies memory.
Reproduce the example in Python
This standard-library example uses a flat list and an offset function. It also builds the two matrix results above. The assertions check the expected value and both axis orders.
from math import prod
shape = (2, 2, 3, 3) # batch, row, column, channel
values = list(range(prod(shape)))
def offset(shape, index):
assert len(shape) == len(index)
result = 0
for length, position in zip(shape, index):
assert 0 <= position < length
result = result * length + position
return result
index = (0, 1, 2, 1)
assert values[offset(shape, index)] == 16
nchw = [values[offset(shape, (b, r, c, k))]
for b in range(2) for k in range(3)
for r in range(2) for c in range(3)]
assert nchw[offset((2, 3, 2, 3), (0, 1, 1, 2))] == 16
matrix = [[0, 1, 2], [3, 4, 5]]
flat = [value for row in matrix for value in row]
reshaped = [flat[start:start + 2] for start in range(0, 6, 2)]
transposed = [list(column) for column in zip(*matrix)]
print("NHWC offset:", offset(shape, index))
print("NCHW offset:", offset((2, 3, 2, 3), (0, 1, 1, 2)))
print("Reshape:", reshaped)
print("Transpose:", transposed)
Expected output:
NHWC offset: 16
NCHW offset: 11
Reshape: [[0, 1], [2, 3], [4, 5]]
Transpose: [[0, 3], [1, 4], [2, 5]]
Check your understanding
Exercise 1. A batch has NHWC shape (5, 8, 10, 3). How many elements does it contain? What shapes result from X[2] and X[2:3]?
Show solution: count elements and retain axes
The batch contains 5 × 8 × 10 × 3 = 1,200 elements. Integer indexing gives X[2] shape (8, 10, 3), with 240 elements. The slice X[2:3] has shape (1, 8, 10, 3), also with 240 elements. The slice retains a batch axis of length one.
Exercise 2. In the explorer, switch to channels first. Where does the original value X[1, 0, 2, 2] move? Could a plain reshape guarantee that same location-to-value relationship?
Show solution: follow the named axes
The original value is 26. Its new index is Y[1, 2, 0, 2], following batch, channel, row, column. The new flat offset is 32. A row-major reshape keeps the original flat sequence, so index [1, 2, 0, 2] would contain value 32. The axis permutation preserves the intended relationship.
For a square matrix, eigenvalues describe how it acts on particular vector directions. That question adds mathematical structure beyond counting axes and elements.
Sources and further study
- TensorFlow: Introduction to Tensors, for array terminology, shape, and order.
- NumPy: Indexing on ndarrays, for integer indexing and basic slices.
- NumPy: reshape, for element order and copy behavior.
- NumPy: transpose, for explicit axis permutations.
- NPTEL: Mathematical Preliminaries, transformation laws, for the physical tensor definition under a change of basis.