Gradient descent
In mathematics, gradient descent (also often called steepest descent) is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent.
- Ability105616246
- Abstraction100002137
- Act100030358
- Activity100407535
- Algorithm105847438
- Cognition100023271
- Event100029378
- Know-how105616786
- Method105660268
- Procedure101023820
- PsychologicalFeature100023100
- Rule105846932
- software
- WikicatFirstOrderMethods
- WikicatGradientMethods
- WikicatOptimizationAlgorithmsAndMethods
- YagoPermanentlyLocatedEntity
- Comment
- enIn mathematics, gradient descent (also often called steepest descent) is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent.
- Depiction
.jpg)
- Has abstract
- enIn mathematics, gradient descent (also often called steepest descent) is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent. Gradient descent is generally attributed to Augustin-Louis Cauchy, who first suggested it in 1847. Jacques Hadamard independently proposed a similar method in 1907. Its convergence properties for non-linear optimization problems were first studied by Haskell Curry in 1944, with the method becoming increasingly well-studied and used in the following decades.
- Hypernym
- Algorithm
- Is primary topic of
- Gradient descent
- Label
- enGradient descent
- Link from a Wikipage to an external page
- web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf%23page=471
- neuralnetworksanddeeplearning.com/chap1.html%23learning_with_gradient_descent
- www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/gradient-and-directional-derivatives/v/gradient
- www.google.com/books/edition/An_Introduction_to_Optimization/iD5s0iKXHP8C%3Fhl=en&gbpv=1&pg=PA131
- ghostarchive.org/varchive/youtube/20211211/IHZwWFHWa-w
- codingplayground.blogspot.it/2013/05/learning-linear-regression-with.html
- web.archive.org/web/20171016173155/https:/www.youtube.com/watch%3Fv=IHZwWFHWa-w
- www.youtube.com/watch%3Fv=IHZwWFHWa-w&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&index=2
- Link from a Wikipage to another Wikipage
- Accelerated gradient method
- Accuracy
- Algorithm
- Artificial neural network
- Augustin-Louis Cauchy
- Backpropagation
- Backtracking line search
- Big O notation
- Bowl (vessel)
- Bregman divergence
- Broyden–Fletcher–Goldfarb–Shanno algorithm
- Category:First order methods
- Category:Gradient methods
- Category:Mathematical optimization
- Category:Optimization algorithms and methods
- Cauchy-Schwarz inequality
- Concentric circles
- Condition number
- Conjugate gradient
- Conjugate gradient method
- Constraint (mathematics)
- Contour line
- Convergent series
- Convex function
- Convex programming
- Curvature
- Davidon–Fletcher–Powell formula
- Defined and undefined
- Delta rule
- Differentiable function
- Differentiation (mathematics)
- Eigenvalues
- Euclidean norm
- Euler's method
- Fast gradient method
- Fast proximal gradient method
- File:Gradient descent.svg
- File:Gradient Descent Example Nonlinear Equations.gif
- File:Gradient Descent in 2D.webm
- File:Okanogan-Wenatchee National Forest, morning fog shrouds trees (37171636495).jpg
- File:Steepest descent.png
- Forward–backward algorithm
- Fréchet derivative
- Function space
- Gauss–Newton algorithm
- Gradient
- Gradient flow
- Haskell Curry
- Hessian matrix
- Hill climbing
- Iterative algorithm
- Jacobian matrix
- Jacques Hadamard
- Learning rate
- Limited-memory BFGS
- Linear combination
- Linear least squares
- Line search
- Lipschitz continuity
- Local maximum
- Local minimum
- Loss function
- Mathematical optimization
- Mirror descent
- Monotonic function
- Multi-variable function
- Nelder–Mead method
- Newton's method in optimization
- Newtonian dynamics
- Nonlinear equation
- Optimized gradient method
- Ordinary differential equations
- Orthogonal
- Philip Wolfe (mathematician)
- Positive-definite matrix
- Preconditioner
- Preconditioning
- Projection (linear algebra)
- Proximal gradient method
- Quantum annealing
- Rprop
- Saddle point
- Slope
- Stochastic gradient descent
- Symmetric matrix
- TFNP
- Variational inequality
- Viscous
- Wolfe conditions
- YouTube
- Yurii Nesterov
- SameAs
- Algorisme del gradient descendent
- Algorithme du gradient
- Discesa del gradiente
- FGLd
- Gradient descent
- Gradient descent
- Gradientenverfahren
- Gradientinis nusileidimas
- Gradientní sestup
- m.01cmhh
- Metoda gradientu prostego
- Método do gradiente
- Penurunan gradien
- Q1199743
- Suy giảm độ dốc
- Алгоритам опадајућег градијента
- Градиентный спуск
- Градієнтний спуск
- خوارزمية أصل التدرج
- گرادیان کاهشی
- ഗ്രേഡിയന്റ് ഡിസെന്റ്
- การเคลื่อนลงตามความชัน
- 最急降下法
- 梯度下降法
- 경사 하강법
- Subject
- Category:First order methods
- Category:Gradient methods
- Category:Mathematical optimization
- Category:Optimization algorithms and methods
- Thumbnail
- WasDerivedFrom
- Gradient descent?oldid=1119406164&ns=0
- WikiPageLength
- 34717
- Wikipage page ID
- 201489
- Wikipage revision ID
- 1119406164
- WikiPageUsesTemplate
- Template:About
- Template:Cbignore
- Template:Cite book
- Template:Cite web
- Template:Commons category
- Template:Differentiable computing
- Template:Div col
- Template:Div col end
- Template:EquationNote
- Template:EquationRef
- Template:Machine learning
- Template:NumBlk
- Template:Optimization algorithms
- Template:Reflist
- Template:Short description
