.. _f-sgd:

module ``optim.sgd``
====================





.. inheritance-diagram:: mlstatpy.optim.sgd


Short summary
+++++++++++++

module ``mlstatpy.optim.sgd``

Implements simple stochastic gradient optimisation.
It is inspired from `_stochastic_optimizers.py
<https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/
neural_network/_stochastic_optimizers.py>`_.


:githublink:`%|py|8`




Classes
+++++++

+-----------------------------------------------------------+------------------------------------------------------+
| class                                                     | truncated documentation                              |
+===========================================================+======================================================+
| :class:`BaseOptimizer <mlstatpy.optim.sgd.BaseOptimizer>` | Base stochastic gradient descent optimizer.          |
+-----------------------------------------------------------+------------------------------------------------------+
| :class:`SGDOptimizer <mlstatpy.optim.sgd.SGDOptimizer>`   | Stochastic gradient descent optimizer with momentum. |
+-----------------------------------------------------------+------------------------------------------------------+










Methods
+++++++

+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| method                                                                                          | truncated documentation                                                                   |
+=================================================================================================+===========================================================================================+
| :py:meth:`__init__ <mlstatpy.optim.sgd.BaseOptimizer.__init__>`                                 |                                                                                           |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :py:meth:`__init__ <mlstatpy.optim.sgd.SGDOptimizer.__init__>`                                  |                                                                                           |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :py:meth:`_display_progress <mlstatpy.optim.sgd.BaseOptimizer._display_progress>`               | Displays training progress.                                                               |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :py:meth:`_display_progress <mlstatpy.optim.sgd.SGDOptimizer._display_progress>`                | Displays training progress.                                                               |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :py:meth:`_evaluate_early_stopping <mlstatpy.optim.sgd.BaseOptimizer._evaluate_early_stopping>` |                                                                                           |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :py:meth:`_evaluate_early_stopping <mlstatpy.optim.sgd.SGDOptimizer._evaluate_early_stopping>`  |                                                                                           |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :py:meth:`_get_updates <mlstatpy.optim.sgd.BaseOptimizer._get_updates>`                         |                                                                                           |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :py:meth:`_get_updates <mlstatpy.optim.sgd.SGDOptimizer._get_updates>`                          | Gets the values used to update params with given gradients.                               |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :py:meth:`_regularize_gradient <mlstatpy.optim.sgd.BaseOptimizer._regularize_gradient>`         | Applies regularization.                                                                   |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :py:meth:`_regularize_gradient <mlstatpy.optim.sgd.SGDOptimizer._regularize_gradient>`          | Applies regularization.                                                                   |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :meth:`iteration_ends <mlstatpy.optim.sgd.BaseOptimizer.iteration_ends>`                        | Performs update to learning rate and potentially other states at the end of an iteration. |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :meth:`iteration_ends <mlstatpy.optim.sgd.SGDOptimizer.iteration_ends>`                         | Performs updates to learning rate and potential other states at the end of an iteration.  |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :meth:`loss_regularization <mlstatpy.optim.sgd.BaseOptimizer.loss_regularization>`              |                                                                                           |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :meth:`loss_regularization <mlstatpy.optim.sgd.SGDOptimizer.loss_regularization>`               |                                                                                           |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :meth:`train <mlstatpy.optim.sgd.BaseOptimizer.train>`                                          | Optimizes the coefficients.                                                               |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :meth:`train <mlstatpy.optim.sgd.SGDOptimizer.train>`                                           | Optimizes the coefficients.                                                               |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :meth:`update_coef <mlstatpy.optim.sgd.BaseOptimizer.update_coef>`                              | Updates coefficients with given gradient.                                                 |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
| :meth:`update_coef <mlstatpy.optim.sgd.SGDOptimizer.update_coef>`                               | Updates coefficients with given gradient.                                                 |
+-------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------+


Documentation
+++++++++++++

.. automodule:: mlstatpy.optim.sgd
    :members:
    :special-members: __init__
    :show-inheritance: