国外一本比较优秀的优化理论的书的习题全解，作者RAO，全书主要讲工程优化的理论和算法。
目录：
1 Introduction to Optimization 1
1.1 Introduction 1
1.2 Historical Development 3
1.3 Engineering Applications of Optimization 5
1.4 Statement of an Optimization Problem 6
1.4.1 Design Vector 6
1.4.2 Design Constraints 7
1.4.3 Constraint Surface 8
1.4.4 Objective Function 9
1.4.5 Objective Function Surfaces 9
1.5 Classification of Optimization Problems 14
1.5.1 Classification Based on the Existence of Constraints 14
1.5.2 Classification Based on the Nature of the Design Variables 15
1.5.3 Classification Based on the Physical Structure of the Problem 16
1.5.4 Classification Based on the Nature of the Equations Involved 19
1.5.5 Classification Based on the Permissible Values of the Design Variables 28
1.5.6 Classification Based on the Deterministic Nature of the Variables 29
1.5.7 Classification Based on the Separability of the Functions 30
1.5.8 Classification Based on the Number of Objective Functions 32
1.6 Optimization Techniques 35
1.7 Engineering Optimization Literature 35
1.8 Solution of Optimization Problems Using MATLAB 36
References and Bibliography 39
Review Questions 45
Problems 46
2 Classical Optimization Techniques 63
2.1 Introduction 63
2.2 Single-Variable Optimization 63
2.3 Multivariable Optimization with No Constraints 68
2.3.1 Semidefinite Case 73
2.3.2 Saddle Point 73
2.4 Multivariable Optimization with Equality Constraints 75
2.4.1 Solution by Direct Substitution 76
2.4.2 Solution by the Method of Constrained Variation 77
2.4.3 Solution by the Method of Lagrange Multipliers 85
2.5 Multivariable Optimization with Inequality Constraints 93
2.5.1 Kuhn–Tucker Conditions 98
2.5.2 Constraint Qualification 98
2.6 Convex Programming Problem 104
References and Bibliography 105
Review Questions 105
Problems 106
3 Linear Programming I: Simplex Method 119
3.1 Introduction 119
3.2 Applications of Linear Programming 120
3.3 Standard Form of a Linear Programming Problem 122
3.4 Geometry of Linear Programming Problems 124
3.5 Definitions and Theorems 127
3.6 Solution of a System of Linear Simultaneous Equations 133
3.7 Pivotal Reduction of a General System of Equations 135
3.8 Motivation of the Simplex Method 138
3.9 Simplex Algorithm 139
3.9.1 Identifying an Optimal Point 140
3.9.2 Improving a Nonoptimal Basic Feasible Solution 141
3.10 Two Phases of the Simplex Method 150
3.11 MATLAB Solution of LP Problems 156
References and Bibliography 158
Review Questions 158
Problems 160
4 Linear Programming II: Additional Topics and Extensions 177
4.1 Introduction 177
4.2 Revised Simplex Method 177
4.3 Duality in Linear Programming 192
4.3.1 Symmetric Primal–Dual Relations 192
4.3.2 General Primal–Dual Relations 193
4.3.3 Primal–Dual Relations When the Primal Is in Standard Form 193
4.3.4 Duality Theorems 195
4.3.5 Dual Simplex Method 195
4.4 Decomposition Principle 200
4.5 Sensitivity or Postoptimality Analysis 207
4.5.1 Changes in the Right-Hand-Side Constants bi 208
4.5.2 Changes in the Cost Coefficients cj 212
4.5.3 Addition of New Variables 214
4.5.4 Changes in the Constraint Coefficients aij 215
4.5.5 Addition of Constraints 218
4.6 Transportation Problem 220
4.7 Karmarkar’s Interior Method 222
4.7.1 Statement of the Problem 223
4.7.2 Conversion of an LP Problem into the Required Form 224
4.7.3 Algorithm 226
4.8 Quadratic Programming 229
4.9 MATLAB Solutions 235
References and Bibliography 237
Review Questions 239
Problems 239
5 Nonlinear Programming I: One-Dimensional Minimization Methods 248
5.1 Introduction 248
5.2 Unimodal Function 253
ELIMINATION METHODS 254
5.3 Unrestricted Search 254
5.3.1 Search with Fixed Step Size 254
5.3.2 Search with Accelerated Step Size 255
5.4 Exhaustive Search 256
5.5 Dichotomous Search 257
5.6 Interval Halving Method 260
5.7 Fibonacci Method 263
5.8 Golden Section Method 267
5.9 Comparison of Elimination Methods 271
INTERPOLATION METHODS 271
5.10 Quadratic Interpolation Method 273
5.11 Cubic Interpolation Method 280
5.12 Direct Root Methods 286
5.12.1 Newton Method 286
5.12.2 Quasi-Newton Method 288
5.12.3 Secant Method 290
5.13 Practical Considerations 293
5.13.1 How to Make the Methods Efficient and More Reliable 293
5.13.2 Implementation in Multivariable Optimization Problems 293
5.13.3 Comparison of Methods 294
5.14 MATLAB Solution of One-Dimensional Minimization Problems 294
References and Bibliography 295
Review Questions 295
Problems 296
6 Nonlinear Programming II: Unconstrained Optimization Techniques 301
6.1 Introduction 301
6.1.1 Classification of Unconstrained Minimization Methods 304
6.1.2 General Approach 305
6.1.3 Rate of Convergence 305
6.1.4 Scaling of Design Variables 305
DIRECT SEARCH METHODS 309
6.2 Random Search Methods 309
6.2.1 Random Jumping Method 311
6.2.2 Random Walk Method 312
6.2.3 Random Walk Method with Direction Exploitation 313
6.2.4 Advantages of Random Search Methods 314
6.3 Grid Search Method 314
6.4 Univariate Method 315
6.5 Pattern Directions 318
6.6 Powell’s Method 319
6.6.1 Conjugate Directions 319
6.6.2 Algorithm 323
6.7 Simplex Method 328
6.7.1 Reflection 328
6.7.2 Expansion 331
6.7.3 Contraction 332
INDIRECT SEARCH (DESCENT) METHODS 335
6.8 Gradient of a Function 335
6.8.1 Evaluation of the Gradient 337
6.8.2 Rate of Change of a Function along a Direction 338
6.9 Steepest Descent (Cauchy) Method 339
6.10 Conjugate Gradient (Fletcher–Reeves) Method 341
6.10.1 Development of the Fletcher–Reeves Method 342
6.10.2 Fletcher–Reeves Method 343
6.11 Newton’s Method 345
6.12 Marquardt Method 348
6.13 Quasi-Newton Methods 350
6.13.1 Rank 1 Updates 351
6.13.2 Rank 2 Updates 352
6.14 Davidon–Fletcher–Powell Method 354
6.15 Broyden–Fletcher–Goldfarb–Shanno Method 360
6.16 Test Functions 363
6.17 MATLAB Solution of Unconstrained Optimization Problems 365
References and Bibliography 366
Review Questions 368
Problems 370
7 Nonlinear Programming III: Constrained Optimization Techniques 380
7.1 Introduction 380
7.2 Characteristics of a Constrained Problem 380
DIRECT METHODS 383
7.3 Random Search Methods 383
7.4 Complex Method 384
7.5 Sequential Linear Programming 387
7.6 Basic Approach in the Methods of Feasible Directions 393
7.7 Zoutendijk’s Method of Feasible Directions 394
7.7.1 Direction-Finding Problem 395
7.7.2 Determination of Step Length 398
7.7.3 Termination Criteria 401
7.8 Rosen’s Gradient Projection Method 404
7.8.1 Determination of Step Length 407
7.9 Generalized Reduced Gradient Method 412
7.10 Sequential Quadratic Programming 422
7.10.1 Derivation 422
7.10.2 Solution Procedure 425
INDIRECT METHODS 428
7.11 Transformation Techniques 428
7.12 Basic Approach of the Penalty Function Method 430
7.13 Interior Penalty Function Method 432
7.14 Convex Programming Problem 442
7.15 Exterior Penalty Function Method 443
7.16 Extrapolation Techniques in the Interior Penalty Function Method 447
7.16.1 Extrapolation of the Design Vector X 448
7.16.2 Extrapolation of the Function f 450
7.17 Extended Interior Penalty Function Methods 451
7.17.1 Linear Extended Penalty Function Method 451
7.17.2 Quadratic Extended Penalty Function Method 452
7.18 Penalty Function Method for Problems with Mixed Equality and Inequality
Constraints 453
7.18.1 Interior Penalty Function Method 454
7.18.2 Exterior Penalty Function Method 455
7.19 Penalty Function Method for Parametric Constraints 456
7.19.1 Parametric Constraint 456
7.19.2 Handling Parametric Constraints 457
7.20 Augmented Lagrange Multiplier Method 459
7.20.1 Equality-Constrained Problems 459
7.20.2 Inequality-Constrained Problems 462
7.20.3 Mixed Equality–Inequality-Constrained Problems 463
7.21 Checking the Convergence of Constrained Optimization Problems 464
7.21.1 Perturbing the Design Vector 465
7.21.2 Testing the Kuhn–Tucker Conditions 465
7.22 Test Problems 467
7.22.1 Design of a Three-Bar Truss 467
7.22.2 Design of a Twenty-Five-Bar Space Truss 468
7.22.3 Welded Beam Design 470
7.22.4 Speed Reducer (Gear Train) Design 472
7.22.5 Heat Exchanger Design 473
7.23 MATLAB Solution of Constrained Optimization Problems 474
References and Bibliography 476
Review Questions 478
Problems 480
8 Geometric Programming 492
8.1 Introduction 492
8.2 Posynomial 492
8.3 Unconstrained Minimization Problem 493
8.4 Solution of an Unconstrained Geometric Programming Program Using Differential
Calculus 493
8.5 Solution of an Unconstrained Geometric Programming Problem Using
Arithmetic–Geometric Inequality 500
8.6 Primal–Dual Relationship and Sufficiency Conditions in the Unconstrained
Case 501
8.7 Constrained Minimization 508
8.8 Solution of a Constrained Geometric Programming Problem 509
8.9 Primal and Dual Programs in the Case of Less-Than Inequalities 510
8.10 Geometric Programming with Mixed Inequality Constraints 518
8.11 Complementary Geometric Programming 520
8.12 Applications of Geometric Programming 525
References and Bibliography 537
Review Questions 539
Problems 540
9 Dynamic Programming 544
9.1 Introduction 544
9.2 Multistage Decision Processes 545
9.2.1 Definition and Examples 545
9.2.2 Representation of a Multistage Decision Process 546
9.2.3 Conversion of a Nonserial System to a Serial System 548
9.2.4 Types of Multistage Decision Problems 548
9.3 Concept of Suboptimization and Principle of Optimality 549
9.4 Computational Procedure in Dynamic Programming 553
9.5 Example Illustrating the Calculus Method of Solution 555
9.6 Example Illustrating the Tabular Method of Solution 560
9.7 Conversion of a Final Value Problem into an Initial Value Problem 566
9.8 Linear Programming as a Case of Dynamic Programming 569
9.9 Continuous Dynamic Programming 573
9.10 Additional Applications 576
9.10.1 Design of Continuous Beams 576
9.10.2 Optimal Layout (Geometry) of a Truss 577
9.10.3 Optimal Design of a Gear Train 579
9.10.4 Design of a Minimum-Cost Drainage System 579
References and Bibliography 581
Review Questions 582
Problems 583
10 Integer Programming 588
10.1 Introduction 588
INTEGER LINEAR PROGRAMMING 589
10.2 Graphical Representation 589
10.3 Gomory’s Cutting Plane Method 591
10.3.1 Concept of a Cutting Plane 591
10.3.2 Gomory’s Method for All-Integer Programming Problems 592
10.3.3 Gomory’s Method for Mixed-Integer Programming Problems 599
10.4 Balas’ Algorithm for Zero–One Programming Problems 604
INTEGER NONLINEAR PROGRAMMING 606
10.5 Integer Polynomial Programming 606
10.5.1 Representation of an Integer Variable by an Equivalent System of Binary
Variables 607
10.5.2 Conversion of a Zero–One Polynomial Programming Problem into a
Zero–One LP Problem 608
10.6 Branch-and-Bound Method 609
10.7 Sequential Linear Discrete Programming 614
10.8 Generalized Penalty Function Method 619
10.9 Solution of Binary Programming Problems Using MATLAB 624
References and Bibliography 625
Review Questions 626
Problems 627
11 Stochastic Programming 632
11.1 Introduction 632
11.2 Basic Concepts of Probability Theory 632
11.2.1 Definition of Probability 632
11.2.2 Random Variables and Probability Density Functions 633
11.2.3 Mean and Standard Deviation 635
11.2.4 Function of a Random Variable 638
11.2.5 Jointly Distributed Random Variables 639
11.2.6 Covariance and Correlation 640
11.2.7 Functions of Several Random Variables 640
11.2.8 Probability Distributions 643
11.2.9 Central Limit Theorem 647
11.3 Stochastic Linear Programming 647
11.4 Stochastic Nonlinear Programming 652
11.4.1 Objective Function 652
11.4.2 Constraints 653
11.5 Stochastic Geometric Programming 659
References and Bibliography 661
Review Questions 662
Problems 663
12 Optimal Control and Optimality Criteria Methods 668
12.1 Introduction 668
12.2 Calculus of Variations 668
12.2.1 Introduction 668
12.2.2 Problem of Calculus of Variations 669
12.2.3 Lagrange Multipliers and Constraints 675
12.2.4 Generalization 678
12.3 Optimal Control Theory 678
12.3.1 Necessary Conditions for Optimal Control 679
12.3.2 Necessary Conditions for a General Problem 681
12.4 Optimality Criteria Methods 683
12.4.1 Optimality Criteria with a Single Displacement Constraint 683
12.4.2 Optimality Criteria with Multiple Displacement Constraints 684
12.4.3 Reciprocal Approximations 685
References and Bibliography 689
Review Questions 689
Problems 690
13 Modern Methods of Optimization 693
13.1 Introduction 693
13.2 Genetic Algorithms 694
13.2.1 Introduction 694
13.2.2 Representation of Design Variables 694
13.2.3 Representation of Objective Function and Constraints 696
13.2.4 Genetic Operators 697
13.2.5 Algorithm 701
13.2.6 Numerical Results 702
13.3 Simulated Annealing 702
13.3.1 Introduction 702
13.3.2 Procedure 703
13.3.3 Algorithm 704
13.3.4 Features of the Method 705
13.3.5 Numerical Results 705
13.4 Particle Swarm Optimization 708
13.4.1 Introduction 708
13.4.2 Computational Implementation of PSO 709
13.4.3 Improvement to the Particle Swarm Optimization Method 710
13.4.4 Solution of the Constrained Optimization Problem 711
13.5 Ant Colony Optimization 714
13.5.1 Basic Concept 714
13.5.2 Ant Searching Behavior 715
13.5.3 Path Retracing and Pheromone Updating 715
13.5.4 Pheromone Trail Evaporation 716
.......................