Computation of Descent Directions in Stochastic Optimization Problems with Invariant Distributions

One of the main problems in stochastic optimization is the minimization of mean value functions F(x) = Eu(A(ω) x ‐ b(ω)) s.t. x ∈ D, where (A(ω), b(ω)) is a random matrix, u is a loss function on R m and D is a convex subset of R n . If the distribution μ of (A(ω), b(ω)) is invariant with respect to an affine transformation Λ of R n(m+1) , i.e. if Λ(μ) = μ, and if for a given x ∈ R n there exists a vector y = y(x) such that \documentclass{article}\pagestyle{empty}\begin{document}$ \Lambda \left({A,b} \right)\left({\begin{array}{*{20}c} x \\ {- 1} \\ \end{array}} \right) = Ay - b $\end{document} a.s., then F(y) = F(x), and, consequently, d = y ‐ x is a descent direction of F at x, provided that F is not constant on the line segment joining x and y. For many practically important distributions μ the affine transformation Λ, a vector y(x) and therefore also a direction of decrease d = y(x) ‐ x is constructed without using any derivatives of F.