Tensors

If you have reached here, you probably know what a tensor is, and probably have heard many jokes about what a tensor is^[1]. Nevertheless, we are gonna give a brief remainder.

A tensor $T$ of order^[2] $n$ is a multilinear^[3] function between $n$ vector spaces over a field $\mathcal{F}$.

\[T : \mathcal{F}^{\dim(1)} \times \dots \times \mathcal{F}^{\dim(n)} \mapsto \mathcal{F}\]

In layman's terms, you can view a tensor as a linear function that maps a set of vectors to a scalar.

\[T(\mathbf{v}^{(1)}, \dots, \mathbf{v}^{(n)}) = c \in \mathcal{F} \qquad\qquad \forall i, \mathbf{v}^{(i)} \in \mathcal{F}^{\dim(i)}\]

Just like with matrices and vectors, $n$-dimensional arrays of numbers can be used to represent tensors. Furthermore, scalars, vectors and matrices can be viewed as tensors of order 0, 1 and 2, respectively.

The dimensions of the tensors are usually identified with labels and known as tensor indices or just indices. By appropeately fixing the indices in a expression, a lot of different linear algebra operations can be described.

For example, the trace operation...

\[tr(A) = \sum_i A_{ii}\]

... a tranposition of dimensions...

\[A_{ji} = A^T_{ij}\]

... or a matrix multiplication.

\[C_{ik} = \sum_j A_{ij} B_{jk}\]

The Tensor type

In Muscle, a tensor is represented by the Tensor type, which wraps an array and a list of Index. An Index is wrapper type that can fit anything in it.

julia> Index(:i)index<i>
julia> Index(1)index<1>
julia> Index((1,2))index<(1, 2)>

You can create a Tensor by passing an AbstractArray and a Vector or Tuple of Indexs.

julia> t = Tensor(rand(3,5,2), [Index(:i), Index(:j), Index(:k)])3×5×2 Tensor{Float64, 3, Array{Float64, 3}}:
[:, :, 1] =
 0.12971    0.63313   0.347636  0.870716  0.0580387
 0.716052   0.820441  0.698069  0.533624  0.495415
 0.0435087  0.176174  0.834953  0.368355  0.103442

[:, :, 2] =
 0.900947  0.789608   0.251433  0.996797  0.268723
 0.351691  0.0985428  0.617252  0.440079  0.322285
 0.660194  0.679977   0.582922  0.194802  0.40499

julia> Tensor(rand(3,2), [:i, :j])3×2 Tensor{Float64, 2, Matrix{Float64}}:
 0.277648   0.462527
 0.0723528  0.19323
 0.648637   0.139295

Use parent and inds to access the underlying array and indices respectively.

julia> parent(t)3×5×2 Array{Float64, 3}:
[:, :, 1] =
 0.12971    0.63313   0.347636  0.870716  0.0580387
 0.716052   0.820441  0.698069  0.533624  0.495415
 0.0435087  0.176174  0.834953  0.368355  0.103442

[:, :, 2] =
 0.900947  0.789608   0.251433  0.996797  0.268723
 0.351691  0.0985428  0.617252  0.440079  0.322285
 0.660194  0.679977   0.582922  0.194802  0.40499
julia> inds(t)3-element Muscle.ImmutableArray{Index, 1}:
 index<i>
 index<j>
 index<k>

The dimensionality or size of each index can be consulted using the size function.

julia> size(t)(3, 5, 2)
julia> size(t, Index(:j))5
julia> length(t)30

Note that these indices are the ones that really define the dimensions of the tensor and not the order of the array dimensions.

julia> a = Tensor([1 0; 1 0], (:i, :j))2×2 Tensor{Int64, 2, Matrix{Int64}}:
 1  0
 1  0
julia> b = Tensor([1 1; 0 0], (:j, :i))2×2 Tensor{Int64, 2, Matrix{Int64}}:
 1  1
 0  0
julia> a ≈ btrue

This is key for interacting with other tensors.

julia> c = a + b2×2 Tensor{Int64, 2, Matrix{Int64}}:
 2  1
 1  0
julia> parent(a) + parent(b)2×2 Matrix{Int64}:
 2  1
 1  0

As such adjoint doesn't permute the dimensions; it just conjugates the array.

julia> d = Tensor([1im 2im; 3im 4im], (:i, :j))2×2 Tensor{Complex{Int64}, 2, Matrix{Complex{Int64}}}:
 0+1im  0+2im
 0+3im  0+4im
julia> d'2×2 Tensor{Complex{Int64}, 2, Matrix{Complex{Int64}}}:
 0-1im  0-2im
 0-3im  0-4im
julia> conj(d)2×2 Tensor{Complex{Int64}, 2, Matrix{Complex{Int64}}}:
 0-1im  0-2im
 0-3im  0-4im

Indexing

Tensor, as a subtype of AbstractArray, allows direct indexing of the underneath array with getindex/setindex! or the [...] notation.

julia> a[1,1] = 33
julia> a[1,:]2-element Vector{Int64}:
 3
 0

But like explained above, on Tensor you should refer the dimensions by their index label, which Tenet allows in many methods.

julia> a[i=1,j=1]3

Check out that not specifying all the indices is equivalent to using : on the non-specified indices.

julia> a[i=1]2-element Vector{Int64}:
 3
 0
julia> a[i=1,j=:]2-element Vector{Int64}:
 3
 0

Other supported methods are permutedims, selectdim and view.

julia> permutedims(a, [:j, :i])ERROR: ArgumentError: invalid index: :j of type Symbol
julia> selectdim(a, :i, 1)2-element Tensor{Int64, 1, SubArray{Int64, 1, Matrix{Int64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}}:
 3
 0
julia> view(a, :i=>1)2-element Tensor{Int64, 1, SubArray{Int64, 1, Matrix{Int64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}}:
 3
 0