*********************************************************************** * * * PLIS - A LIMITED MEMORY VARIABLE METRIC ALGORITHM FOR * * LARGE-SCALE OPTIMIZATION. * * * *********************************************************************** 1. Introduction: ---------------- The double-precision FORTRAN 77 basic subroutine PLIS is designed to find a close approximation to a local minimum of a nonlinear function F(X) with simple bounds on variables. Here X is a vector of NF variables and F(X) is a smooth function. We suppose that NF is large but the sparsity pattern of the Hessian matrix is not known (or the Hessian matrix is dense). Simple bounds are assumed in the form X(I) unbounded if IX(I) = 0, XL(I) <= X(I) if IX(I) = 1, X(I) <= XU(I) if IX(I) = 2, XL(I) <= X(I) <= XU(I) if IX(I) = 3, XL(I) = X(I) = XU(I) if IX(I) = 5, where 1 <= I <= NF. To simplify user's work, two additional easy to use subroutines are added. They call the basic general subroutine PLIS: PLISU - unconstrained large-scale optimization, PLISS - large-scale optimization with simple bounds. All subroutines contain a description of formal parameters and extensive comments. Furthermore, two test programs TLISU and TLISS are included, which contain several test problems (see e.g. [2]). These test programs serve as examples for using the subroutines, verify their correctness and demonstrate their efficiency. In this short guide, we describe all subroutines which can be called from the user's program. A detailed description of the method is given in [1]. In the description of formal parameters, we introduce a type of the argument that specifies whether the argument must have a value defined on entry to the subroutine (I), whether it is a value which will be returned (O), or both (U), or whether it is an auxiliary value (A). Besides formal parameters, we can use a COMMON /STAT/ block containing statistical information. This block, used in each subroutine has the following form: COMMON /STAT/ NRES,NDEC,NIN,NIT,NFV,NFG,NFH The arguments have the following meaning: Argument Type Significance ---------------------------------------------------------------------- NRES O Positive INTEGER variable that indicates the number of restarts. NDEC O Positive INTEGER variable that indicates the number of matrix decompositions. NIN O Positive INTEGER variable that indicates the number of inner iterations (for solving linear systems). NIT O Positive INTEGER variable that indicates the number of iterations. NFV O Positive INTEGER variable that indicates the number of function evaluations. NFG O Positive INTEGER variable that indicates the number of gradient evaluations. NFH O Positive INTEGER variable that indicates the number of Hessian evaluations. 2. Subroutines PLISU, PLISS: ---------------------------- The calling sequences are CALL PLISU(NF,X,IPAR,RPAR,F,GMAX,IPRNT,ITERM) CALL PLISS(NF,X,IX,XL,XU,IPAR,RPAR,F,GMAX,IPRNT,ITERM) The arguments have the following meaning. Argument Type Significance ---------------------------------------------------------------------- NF I Positive INTEGER variable that specifies the number of variables of the objective function. X(NF) U On input, DOUBLE PRECISION vector with the initial estimate to the solution. On output, the approximation to the minimum. IX(NF) I On input (significant only for PLISS) INTEGER vector containing the simple bounds types: IX(I)=0 - the variable X(I) is unbounded, IX(I)=1 - the lower bound X(I) >= XL(I), IX(I)=2 - the upper bound X(I) <= XU(I), IX(I)=3 - the two side bound XL(I) <= X(I) <= XU(I), IX(I)=5 - the variable X(I) is fixed (given by its initial estimate). XL(NF) I DOUBLE PRECISION vector with lower bounds for variables (significant only for PLISS). XU(NF) I DOUBLE PRECISION vector with upper bounds for variables (significant only for PLISS). IPAR(7) U INTEGER parameters: IPAR(1)=MIT, IPAR(2)=MFV, IPAR(3)-unused, IPAR(4)=IEST, IPAR(5)-unused, IPAR(6)-unused, IPAR(7)=MF. Parameters MIT, MFV, IEST, MF are described in Section 3 together with other parameters of the subroutine PLIS. RPAR(9) U DOUBLE PRECISION parameters: RPAR(1)=XMAX, RPAR(2)=TOLX, RPAR(3)=TOLF, RPAR(4)=TOLB, RPAR(5)=TOLG, RPAR(6)=FMIN, RPAR(7)-unused, RPAR(6)-unused, RPAR(9)-unused. Parameters XMAX, TOLX, TOLF, TOLB, TOLG, FMIN are described in Section 3 together with other parameters of the subroutine PLIS. F O DOUBLE PRECISION value of the objective function at the solution X. GMAX O DOUBLE PRECISION maximum absolute value of a partial derivative of the objective function. IPRNT I INTEGER variable that specifies PRINT: IPRNT= 0 - print is suppressed, IPRNT= 1 - basic print of final results, IPRNT=-1 - extended print of final results, IPRNT= 2 - basic print of intermediate and final results, IPRNT=-2 - extended print of intermediate and final results. ITERM O INTEGER variable that indicates the cause of termination: ITERM= 1 - if |X - XO| was less than or equal to TOLX in two subsequent iterations, ITERM= 2 - if |F - FO| was less than or equal to TOLF in two subsequent iterations, ITERM= 3 - if F is less than or equal to TOLB, ITERM= 4 - if GMAX is less than or equal to TOLG, ITERM= 6 - if termination criterion was not satisfied, but the solution is probably acceptable, ITERM=11 - if NIT exceeded MIT, ITERM=12 - if NFV exceeded MFV, ITERM< 0 - if the method failed. The subroutines PLISU, PLISS require the user supplied subroutines OBJ and DOBJ that define the objective function and its gradient and have the form SUBROUTINE OBJ(NF,X,F) SUBROUTINE DOBJ(NF,X,G) The arguments of the user supplied subroutines have the following meaning. Argument Type Significance ---------------------------------------------------------------------- NF I Positive INTEGER variable that specifies the number of variables of the objective function. X(NF) I DOUBLE PRECISION an estimate to the solution. F O DOUBLE PRECISION value of the objective function at the point X. G(NF) O DOUBLE PRECISION gradient of the objective function at the point X. 3. Subroutine PLIS: ------------------- This general subroutine is called from all subroutines described in Section 2. The calling sequence is CALL PLIS(NF,NB,X,IX,XL,XU,GF,S,XO,GO,UO,VO,XMAX,TOLX,TOLF,TOLB, & TOLG,FMIN,GMAX,F,MIT,MFV,IEST,MF,IPRNT,ITERM) The arguments NF, NB, X, IX, XL, XU, GMAX, F, IPRNT, ITERM, have the same meaning as in Section 2. Other arguments have the following meaning: Argument Type Significance ---------------------------------------------------------------------- GF(NF) A DOUBLE PRECISION gradient of the objective function. S(NF) A DOUBLE PRECISION direction vector. XO(NF*MF) A DOUBLE PRECISION array which contains increments of variables. GO(NF*MF) A DOUBLE PRECISION array which contains increments of gradients. UO(MF) A DOUBLE PRECISION Auxiliary array. VO(MF) A DOUBLE PRECISION Auxiliary array. XMAX U DOUBLE PRECISION maximum stepsize; the choice XMAX=0 causes that the default value 1.0D+16 will be taken. TOLX U DOUBLE PRECISION tolerance for the change of the coordinate vector X; the choice TOLX=0 causes that the default value TOLX=1.0D-16 will be taken. TOLF U DOUBLE PRECISION tolerance for the change of function values; the choice TOLF=0 causes that the default value TOLF=1.0D-14 will be taken. TOLB U DOUBLE PRECISION minimum acceptable function value; the choice TOLB=0 causes that the default value TOLB=FMIN+1.0D-16 will be taken. TOLG U DOUBLE PRECISION tolerance for the Lagrangian function gradient; the choice TOLG=0 causes that the default value TOLG=1.0D-6 will be taken. FMIN U DOUBLE PRECISION lower bound for the minimum function value. It is significant only if IEST=1. If IEST=0, the default value FMIN=-1.0D+60 will be taken. MIT U INTEGER variable that specifies the maximum number of iterations; the choice MIT=0 causes that the default value 9000 will be taken. MFV U INTEGER variable that specifies the maximum number of function evaluations; the choice MFV=0 causes that the default value 9000 will be taken. IEST I INTEGER estimation of the minimum functiom value for the line search: IEST=0 - estimation is not used, IEST=1 - lower bound FMIN is used as an estimation for the minimum function value. MF U The number of limited-memory variable metric updates in each iteration (they use 2*MF stored vectors). The choice MF=0 causes that the default value MF=5 will be taken. The choice of parameter XMAX can be sensitive in many cases. First, the objective function can be evaluated only in a relatively small region (if it contains exponentials) so that the maximum stepsize is necessary. Secondly, the problem can be very ill-conditioned far from the solution point so that large steps can be unsuitable. Finally, if the problem has more local solutions, a suitably chosen maximum stepsize can lead to obtaining a better local solution. The subroutine PLIS requires the user supplied subroutines OBJ and DOBJ which are described in Section 2. 4. Verification of the subroutines: ----------------------------------- Subroutine PLISU can be verified and tested using the program TLISU. This program calls the subroutines TIUD14 (initiation), TFFU14 (function evaluation) and TFGU14 (gradient evaluation) containing 22 unconstrained test problems with at most 1000 variables [2]. The results obtained by the program TLISU on a PC computer with Microsoft Power Station Fortran compiler have the following form. NIT= 5077 NFV= 5578 NFG= 5578 F= 0.127201710E-14 G= 0.257E-06 ITERM= 4 NIT= 457 NFV= 501 NFG= 501 F= 14.9944763 G= 0.230E-05 ITERM= 2 NIT= 225 NFV= 248 NFG= 248 F= 0.153527129E-12 G= 0.419E-06 ITERM= 4 NIT= 134 NFV= 145 NFG= 145 F= 269.499543 G= 0.975E-06 ITERM= 4 NIT= 25 NFV= 27 NFG= 27 F= 0.744623638E-12 G= 0.349E-06 ITERM= 4 NIT= 31 NFV= 33 NFG= 33 F= 0.112725896E-10 G= 0.805E-06 ITERM= 4 NIT= 44 NFV= 50 NFG= 50 F= 335.137433 G= 0.821E-06 ITERM= 4 NIT= 30 NFV= 35 NFG= 35 F= 761774.954 G= 0.143E-03 ITERM= 2 NIT= 13 NFV= 16 NFG= 16 F= 316.436141 G= 0.400E-06 ITERM= 4 NIT= 2529 NFV= 2616 NFG= 2616 F= -128.410000 G= 0.151E-04 ITERM= 2 NIT= 142 NFV= 165 NFG= 165 F= 10.7765879 G= 0.713E-06 ITERM= 4 NIT= 276 NFV= 293 NFG= 293 F= 982.273617 G= 0.138E-04 ITERM= 2 NIT= 7 NFV= 8 NFG= 8 F= 0.297153760E-16 G= 0.768E-08 ITERM= 3 NIT= 10 NFV= 12 NFG= 12 F= 0.128730000E-08 G= 0.950E-06 ITERM= 4 NIT= 2724 NFV= 2827 NFG= 2827 F= 1.92401599 G= 0.955E-06 ITERM= 4 NIT= 210 NFV= 225 NFG= 225 F= -427.404476 G= 0.182E-04 ITERM= 2 NIT= 1284 NFV= 1315 NFG= 1315 F=-0.379921090E-01 G= 0.753E-06 ITERM= 4 NIT= 2037 NFV= 2094 NFG= 2094 F=-0.245741193E-01 G= 0.719E-06 ITERM= 4 NIT= 1953 NFV= 1998 NFG= 1998 F= 59.5986241 G= 0.252E-05 ITERM= 2 NIT= 2308 NFV= 2364 NFG= 2364 F= -1.00013520 G= 0.916E-06 ITERM= 4 NIT= 3314 NFV= 3395 NFG= 3395 F= 2.13866377 G= 0.997E-06 ITERM= 4 NIT= 1583 NFV= 1622 NFG= 1622 F= 1.00000000 G= 0.816E-06 ITERM= 4 NITER =24413 NFVAL =25567 NGVAL =25567 NSUCC = 22 TIME= 0:00:04.71 The rows corresponding to individual test problems contain the number of iterations NIT, the number of function evaluations NFV, the number of gradient evaluations NFG, the final value of the objective function F, the norm of gradient G and the cause of termination ITERM. Subroutine PLISS can be verified and tested using the program TLISS. This program calls the subroutines TIUD14 (initiation), TFFU14 (function evaluation), TFGU14 (gradient evaluation) containing 22 box constrained test problems with at most 1000 variables [2]. The results obtained by the program TLISS on a PC computer with Microsoft Power Station Fortran compiler have the following form. NIT= 5425 NFV= 5923 NFG= 5923 F= 0.118621730E-13 G= 0.246E-05 ITERM= 2 NIT= 1966 NFV= 2245 NFG= 2245 F= 3926.45961 G= 0.512E-04 ITERM= 2 NIT= 164 NFV= 180 NFG= 180 F= 0.379502608E-09 G= 0.875E-06 ITERM= 4 NIT= 68 NFV= 76 NFG= 76 F= 269.522686 G= 0.514E-06 ITERM= 4 NIT= 25 NFV= 27 NFG= 27 F= 0.744623638E-12 G= 0.349E-06 ITERM= 4 NIT= 31 NFV= 33 NFG= 33 F= 0.112725896E-10 G= 0.805E-06 ITERM= 4 NIT= 35 NFV= 39 NFG= 39 F= 337.722479 G= 0.438E-06 ITERM= 4 NIT= 51 NFV= 55 NFG= 55 F= 761925.725 G= 0.213E-03 ITERM= 2 NIT= 504 NFV= 506 NFG= 506 F= 428.056916 G= 0.441E-06 ITERM= 4 NIT= 1216 NFV= 1274 NFG= 1274 F= -81.3445208 G= 0.193E-04 ITERM= 2 NIT= 13 NFV= 23 NFG= 23 F= 96517.2947 G= 0.122E-08 ITERM= 4 NIT= 78 NFV= 94 NFG= 94 F= 4994.21410 G= 0.179E-05 ITERM= 2 NIT= 7 NFV= 8 NFG= 8 F= 0.297153760E-16 G= 0.768E-08 ITERM= 3 NIT= 10 NFV= 12 NFG= 12 F= 0.128730000E-08 G= 0.950E-06 ITERM= 4 NIT= 2724 NFV= 2827 NFG= 2827 F= 1.92401599 G= 0.955E-06 ITERM= 4 NIT= 194 NFV= 201 NFG= 201 F= -427.391653 G= 0.202E-04 ITERM= 2 NIT= 1284 NFV= 1315 NFG= 1315 F=-0.379921090E-01 G= 0.753E-06 ITERM= 4 NIT= 2037 NFV= 2094 NFG= 2094 F=-0.245741193E-01 G= 0.719E-06 ITERM= 4 NIT= 1637 NFV= 1687 NFG= 1687 F= 1654.94525 G= 0.105E-04 ITERM= 2 NIT= 2465 NFV= 2528 NFG= 2528 F= -1.00013520 G= 0.872E-06 ITERM= 4 NIT= 1835 NFV= 1876 NFG= 1876 F= 2.41354873 G= 0.854E-06 ITERM= 4 NIT= 1515 NFV= 1559 NFG= 1559 F= 1.00000000 G= 0.892E-06 ITERM= 4 NITER =23284 NFVAL =24582 NGVAL =24582 NSUCC = 22 TIME= 0:00:05.11 References: ----------- [1] Luksan L., Matonoha C., Vlcek J.: LSA: Algorithms for large-scale unconstrained and box constrained optimization. Research Report V-896, Institute of Computer Science, Academy of Sciences of the Czech Republic, Prague, Czech Republic, 2004. [2] Luksan L., Vlcek J.: Sparse and partially separable test problems for unconstrained and equality constrained optimization. Research Report V-767, Institute of Computer Science, Academy of Sciences of the Czech Republic, Prague, Czech Republic, 1998.