Some tips about the optimization techniques in all kinds of common questions.
linear/nonlinear Regression and find optimal point
Least mean square regression
最小二乘方法
Linear Programming
线性规划
Bayesian optimization
example
https://github.com/ytopt-team/ytopt
Matthieu’s work
Workflow configuration things
This approach is popular for the parameter tunning
Gradient descent
用在DL中的optimizer的主要作用就是通过迭代的方式不断地调整input parameter，然后找到一个最小的loss value。
This vedio provides a good introducetion
How to start from the most general gradient decent to the more recent Adam approach.
