[jira] [Created] (FLINK-1994) Add stepsize calculation schemes to SGD

[Shang Yuanchun (Jira)]

Till Rohrmann created FLINK-1994:
------------------------------------

             Summary: Add stepsize calculation schemes to SGD
                 Key: FLINK-1994
                 URL: https://issues.apache.org/jira/browse/FLINK-1994
             Project: Flink
          Issue Type: Improvement
          Components: Machine Learning Library
            Reporter: Till Rohrmann
            Priority: Minor


The current SGD implementation uses as gain for the weight updates the formula {{stepsize/sqrt(iterationNumber)}}. It would be good to make the gain calculation configurable and to provide different strategies for that. For example:

* stepsize/(1 + iterationNumber)
* stepsize*(1 + regularization * stepsize * iterationNumber)^(-3/4)

See also how to properly select the gains [1].

Resources:
[1] http://arxiv.org/pdf/1107.2490.pdf




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)