WTF is a Tensor?!?

A tensor is a container which can house
c
comments

When we represent
Image source

 
Scalar

A single number is what constitutes a scalar. A scalar is a 0-dimensional (0D) tensor. It, thus, has 0 axes, and is of rank 0 (tensor-speak for 'number of axes').

And this is where the nuance comes in: though a single number can be expressed as a tensor, this doesn't mean it should be, or that in generally is. There is good reason to be able to treat them as such (which will become evident when we discuss tensor operations), but as a storage mechanism, this ability can be confounding.

Numpy's multidimensional array ndarray is used below to create the example constructs discussed. Recall that the ndim attribute of the multidimensional array returns the number of array dimensions.

import numpy as np

x = np.array(42)
print(x)
print('A scalar is of rank %d' %(x.ndim))


42
A scalar is of rank 0


 
Vector

A vector is a single dimension (1D) tensor, which you will more commonly hear referred to in computer science as an array. An vector is made up of a series of numbers, has 1 axis, and is of rank 1.

x = np.array([1, 1, 2, 3, 5, 8])
print(x)
print('A vector is of rank %d' %(x.ndim))


[1 1 2 3 5 8]
A vector is of rank 1


 
Matrix

A matrix is a tensor of rank 2, meaning that it has 2 axes. You are familiar with these from all sorts of places, notably what you wrangle your datasets into and feed to your Scikit-learn machine learning models :) A matrix is arranged as a grid of numbers (think rows and columns), and is technically a 2 dimension (2D) tensor.

x = np.array([[1, 4, 7],
              [2, 5, 8],
              [3, 6, 9]])
print(x)
print('A matrix is of rank %d' %(x.ndim))


[[1 4 7]
 [2 5 8]
 [3 6 9]]
A matrix is of rank 2


 
3D Tensor (and higher dimensionality)

While, technically, all of the above constructs are valid tensors, colloquially when we speak of tensors we are generally speaking of the generalization of the concept of a matrix to N ≥ 3 dimensions. We would, then, normally refer only to tensors of 3 dimensions or more as tensors, in order to avoid confusion (referring to the scalar '42' as a tensor would not be beneficial or lend to clarity, generally speaking).

The code below creates a 3D tensor. If we were to pack a series of these into a higher order tensor container, it would be referred to as a 4D tensor; pack those into another order higher, 5D, and so on.

x = np.array([[[1, 4, 7],
               [2, 5, 8],
               [3, 6, 9]],
              [[10, 40, 70],
               [20, 50, 80],
               [30, 60, 90]],
              [[100, 400, 700],
               [200, 500, 800],
               [300, 600, 900]]])
print(x)
print('This tensor is of rank %d' %(x.ndim))


[[[  1   4   7]
  [  2   5   8]
  [  3   6   9]]
 [[ 10  40  70]
  [ 20  50  80]
  [ 30  60  90]]
 [[100 400 700]
  [200 500 800]
  [300 600 900]]]
This tensor is of rank 3


 
What you do with a tensor is your business, though understanding what one is, and its relationship to related numerical container constructs, should now be clear.

 
Related:

  • Boost your data science skills. Learn linear algebra.
  • Deep Learning 101: Demystifying Tensors
  • Building Convolutional Neural Network using NumPy from Scratch