Several Guaranteed Descent Conjugate Gradient Methods for Unconstrained Optimization

This paper investigates a general form of guaranteed descent conjugate gradient methods which satisfies the descent condition and which is strongly convergent whenever the weak Wolfe line search is fulfilled. Moreover, we present several specific guaranteed descent conjugate gradient methods and give their numerical results for large-scale unconstrained optimization. 1. Introduction Consider the following unconstrained optimization problem: where is the -dimensional Euclidean space, is continuously differentiable, and its gradient is available. Conjugate gradient methods are very efficient to solve problem (1) due to their simple iteration and their low memory requirements. For any given starting point , they generate a sequence by the following recursive relation: where , is a step length obtained by means of a one-dimensional search, and is a scalar that characterizes the method. In general, the step length in (2) is obtained by fulfilling the following weak Wolfe conditions [1, 2]: where . And different choices for the scalar in (3) result in different nonlinear conjugate gradient methods. Well-known formulas for are the Fletcher-Reeves (FR), Hestenes-Stiefel (HS), Polak-Ribiére-Polyak (PRP), Dai-Yuan (DY), and Liu-Storey (LS) formulas (see [3], [4], [5], [6], [7], and [8], resp.) and are given by where means the Euclidean norm and . Their corresponding conjugate gradient methods are viewed as basic conjugate gradient methods. Among these basic conjugate gradient methods, the PRP and HS methods perform very similarly and perform better than other basic conjugate gradient methods [9]. While Powell [10] utilized an example illustrating that the PRP and HS methods may cycle without approaching any solution point, then modified versions of the PRP and HS methods were presented by many researchers (see, e.g., [11–16]). The following (sufficient) descent condition, is very important for conjugate gradient methods, so we are particularly interested in the conjugate gradient methods with sufficient descent conditions. Up to now, there are many descent conjugate gradient methods proposed by researchers; please see [12, 16–19] and references therein. One well-known guaranteed descent conjugate gradient method was proposed by Hager and Zhang [16, 20, 21] with The method is designed based on the HS method and satisfies the sufficient descent condition (6) with for any (inexact) line search. In [18], Zhang and Li proposed a general case of the HZ method with where and is a scalar to be specified. It also satisfies the sufficient descent condition (6) with , and