Common mathmetical operations and python examples

Some tips about mathmetical operations and how are they implemented in python library such as numpy.

Set operations

Cartesian product

If A and B are two sets, the cartisian product is defined by (set-builder notation):

$A \times B $ = { (a,b) | a belongs to A and b belongs to B }

The resutls is the set of all ordered pairs (a, b) where a is in A and b is in B.

Array operations

单独的numpy array 更多地用来描述 vector math

一个vector就相当于一个numpy中的array

具体熟练使用的numpy array的操作应该包括如下部分

numpy array转python array

python array转numpy array

生成指定长度的array设置初始值

生成指定长度的元素 每个都有相同的初始值

u = np.empty(Num)
u.fill(4.0)

或者采用

np.ones(n) 的方式进行定义

行列变换

a = np.array([1, 2, 3])
a = a.reshape(-1, 1)
a
array([[1],
       [2],
       [3]])

Length

Normalization

basic operation such as vector sum substract multuply divide

multiple for the elements at the corresponding position

np.array([1,2,3])*np.array([2,3,4])
array([ 2,  6, 12])

Dot Product (inner product)

projection one vector into another vector, the results of the dot product is a value. (remember the projection). For the value, it is a dot b = |a||b|cos(theta), therefore, the a dot b can be used to describe the value of intersection angle between two vector.

consider it in 3d space for vector represented by 2d, just projection one vector onto another one.

you start with two vectors and ends up with a scalar value.

For the complex number, since we need to make sure the energy (magnitude/amplitude) is a postive value, we should use the conjugate transpose instead of the transpose of the original vector when we construct the dot product resutls.

check out this vedio to get more ideas:
https://www.youtube.com/watch?v=Qpm85nC2BBE

Cross Product

the results of the cross product is a vector. The direaction of this vector follows the right handed screw rule. The value of the vector is the area of parallelogram composed of these two vectors. (remember the rotation). there is a particular equation to compute the new vector of the cross product, for example the three dimention vetor times three dimention vector, the equation is a determinant. The first row is the i,j,k, the second row is the x1,y1,z1 and the last row is x2,y2,z2. Then the value of the determinant is a new vector.

bracket representation, since the cross product is so common for some use cases such as in the robotics cases, we also use the A X B = [A]B to represent this, and the [A] can be represented by the 3*3 matrix, the matrix [A] is called the skew-symmetric matrix

If we already know the vector and the rotation axis w than we want to caculate the linear velocity, the idea is to use the -[w]xp why there is a negative mark?

using numpy.cross to express the coresponding results.

Projection

Reflection

tangential vector

numpy init array

https://stackoverflow.com/questions/4535374/initialize-a-numpy-array

初始化array的时候直接使用numpy的方法 比如

numpy.zeros

Return a new array of given shape and type, filled with zeros.

使用tolist的方法可以把array再转化为list

https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.tolist.html

complex vector caculuse (gradient divergency Laplacian curl based on nabla operator)

通俗理解可以参考这个：

https://www.zhihu.com/question/24074028

这几个量都是对于空间中的点而言的，就算是divergency，输入也是空间中的一个点所在位置的vector

https://en.wikipedia.org/wiki/Del

gradient

input is a scalar, output is a vector

gradient会作用在一个scalar field中，gradient操作会在每个点上都求出一个vector，这个vector方向就是scalar field中的值上升最快的方向

具体计算的时候，每个scalar 由空间中的 x,y,z 的位置决定 (assume there is 3d) 这个scarlar value 实质上是空间上的每个点对应的值 v = f (x,y,z) 之后这个函数对每个方向上的分量求偏导数然后每个方向上的分量组成一个vector，就是gradient of f。

Gradient: grad f = ∇ f

divergency

input is vector and the output is a scalar
具体运算 https://en.wikipedia.org/wiki/Divergence

divergency作用在一个vector field中，物理含义是vector field的发散程度
比如在二维空间中，想想一个点周围的极小的圆圈，在每个方向的变化量相加，得到的是总的变化量，也就是发散程度，可以表示为是向正方向变化(向外发散)还是向负方向变化(向内收缩)。

Divergence: div v = ∇ dot v (the v here is a vector)

curl

curl v = ∇ cross dot v (the v here is a vector)

Laplacian

input is the scalar (function value, this value depends on the coordinates, namely the position in the space), the output value is a scalar.

divergency of the gradient, It is usually denoted by the symbols ∇·∇, ∇2

basically, it will combine the gradient and laplacian together. First step is caculating the gradient of the scalar to get a vector, second step is caclulating the divergency of the vector to get the scalar again.

表示偏离平均值的程度，本质上是对空间标量的一种操作，先求该标量的梯度场，再求梯度场的散度。

为什么求scalar field的gradient field的divergency是比较重要的，这个量可以描述field本身的一些properties，中文可以解释为某种驱动力或者势。比如这个例子(https://www.zhihu.com/question/26822364) 通过温度场来理解Laplacian，值如果为正表示发散，说明了热量从这个点流出，反之则是热量流入这个区域。

Matrix operation

More detailed conceptes regarding the linear alg (such as physical mearning of specific operaiton) can be found in the linear_alg_review.md blog

Determinant of matrix (absolute value)

The determinant of matirx has a simple expression which is just a $|A|$ expression, namely adding the absolute operation around the matrix, the reuslt is a scalar value.

Matrix norm

Matrix norm is in the format of $||A||_p$, the typical value of p is 1, 2 and positive infinity. When p is 1, it represents max column sum, when p is 2, it represents max row sum, when p is postive infinity, it represents the largest singular value of A (spectral norm).

basic concept of matrix used in linear algebra

symmetric

conjungate

singular matrix

properties of singular matrix

non-invertible

explain one eigen value corresponds to multiple eigen vectors

对应元素相乘 a[i][j] times b[i][j] = c[i][j]

矩阵乘法运算 c[i][j] 是 a matrix 中的j行 对应乘以b matrix中的j列，然后相加求和。

module of matrix

numpy.matmul

there are still some confusion

Matrix product of two arrays. 使用array表示matrix的时候用这个做乘法，如果本身就是matrix的形式就使用.dot的形式

rank

range

singular matrix

positive definite matrix

null space column space row space left null space

advanced, tensor products and Kronecker product

This is a good vedio that compares these two concepts:

https://www.youtube.com/watch?v=qp_zg_TD0qE

There are some confusion since both the tensor products and the Kronecker product are using the circle corss. One thing that we should be clear is the elements for these two operations.

For Kronecker product, the left and right side of the circle products symble are array, sometimes we can applied it to the two matrix, but be careful about the symbol at this time, we need to view the matrix as an array, which means to write the vector in an form of array.

in numpy, just use the numpy.kron.

This kron product is useful to describe the entanglement of the quantum status of the quantum computing.

For tensor product, although the reaults looks same with the Kronecker product, it do works in different context.

For the tensor operation, the left side and right side of the circle cross are two tensors and the reuslts is a third tensor. For Kronecker product, the left and right of the circle corss are two vector and the result is another vector, even if it can be used to express an matrix.

advanced, tensor learning notes

this is a good series about the tensors:

https://www.youtube.com/watch?v=8ptMTLzV4-I&list=PLJHszsWbB6hrkmmq57lX8BV-o-YIOFsiG

here are some key concepts:

although the quantum computing looks fancy, it is quite straightforward to explain it mathematically, for the superposition, it is just a fancy way to describe the linear compination. For the quantum entanglement, it is just the tensor productr of two (tensors) geometries.

defination of the tensor: there are multiple defination about the tensor, such as the array defination, coordinates defination, and abstract defination. The one that I could figure out quickly but not most accurately is the array defination, basically, the tensor is a multidimentional array. The scarlar can be rank tensor, the vector can be rank 1 tensor, and the matrix can be rank 2 tensor, the 3d grid/mesh is a rank 3 tensor. But this defination lose some key geometry properties. Try to dive into the coordinates defination when it is necessary.

forward backward transformation: there are two coordinates system can be used to represents one tensor, how to tranform from the one coordinates system to another coordinates system. what is the transformation matrix here.

covector: a function that takes a vector and produce a scalar/number, one typical example of the covector is the first row vector when thers is a row vector times the column vector.

solving linear system

svd

deeppeset descent

conjugate descent

其他

定义数据的类型以及memory layout
由于python自身的性质可以在数据初始化的时候不定义数据的格式，常常让人有一种“自由”的感觉 但是有的时候这种“自由”反而容易给人带来不容易发现的bug，比如下面这个case：

本来是想写一个对矩阵进行单位化的操作，就是每列上的元素的平方和相加起来为1，于是就这些写

import numpy as np

a = np.array([
    [1,0,0],
    [1,4,0],
    [1,0,5],
    ]
)


for i in range(3):
    norma = np.linalg.norm(a[:,i])
    normdata = a[:,i]/norma
    a[:,i]=normdata
  

print a

但是打印出来的结果比较奇怪

[[0 0 0]
 [0 1 0]
 [0 0 1]]

后来同学提醒是数据格式有问题，修改之后按照如下形式

import numpy as np

def selfnorm(a):

    for i in range(3):
        norma = np.linalg.norm(a[:,i])
        normdata = a[:,i]/norma
        a[:,i]=normdata
    
    return a
    
a = np.array([
    [1.0,0.0,0.0],
    [1.0,4.0,0.0],
    [0.0,0.0,5.0],
    ],dtype=np.float32

)

print selfnorm(a)

按照上面这样的操作在定义a的时候要加上数据类型，这样就不会被弄成默认的int类型

reference

334 class notes (确保334的内容涵盖在这个note中)

https://orionquest.github.io/Numacom/

fluid engine development 的数学部分

sdf 函数

https://zhuanlan.zhihu.com/p/26217154

https://blogs.mathworks.com/videos/2010/05/05/set-up-finding-closest-point-on-a-surface/

如何直观形象的理解梯度，散度，旋度？

https://www.zhihu.com/question/24074028

不错的书

linear algebra done right

http://148.206.53.84/tesiuami/S_pdfs/Linear%20Algebra%20Done%20Right.pdf

congugate in painless way

painless conjugate descent

good vedio for linear algebra
https://www.youtube.com/watch?v=LyGKycYT2v0&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab&index=10&t=0s