### Unformatted Attachment Preview

Math 171A: Linear Programming
Lecture 1
Introduction to Optimization
Philip E. Gill
© 2021
http://ccom.ucsd.edu/~peg/math171a
Monday, January 4th, 2021
Overview of Math 171A
Instructor
Office Hours
Zoom ID
Philip Gill
Mon & Wed 1:30p–3p
910 9807 9812
Instructor
Office Hours
Zoom ID
Tuesday Section(s)
Ziyan Zhu
Mon & Wed 8a–10a
958 2735 51751
A01, A02 (4p, 5p)
Minxin Zhang
Mon & Wed 6p–8p
916 1560 5957
A03, A04 (6p, 7p)
Jeb Runnoe
Tue 4p–6p
967 0503 1179
A05
Songyu He
Mon & Wed6 4p–6p
860 5312 070
A06, A07 (6p, 7p)
(6p
)
No discussions the first week!
Class webpage: http://ccom.ucsd.edu/~peg/math171a
1
password “math171a”
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
2/37
Overview of Math 171A
• The lectures will be recorded and uploaded to Canvas “later in the day”.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
3/37
Overview of Math 171A
• The lectures will be recorded and uploaded to Canvas “later in the day”.
• Lecture slides are available for download on Canvas and the class website.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
3/37
Overview of Math 171A
• The lectures will be recorded and uploaded to Canvas “later in the day”.
• Lecture slides are available for download on Canvas and the class website.
• Usually, the slides will be available the evening before the lecture.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
3/37
Overview of Math 171A
• The lectures will be recorded and uploaded to Canvas “later in the day”.
• Lecture slides are available for download on Canvas and the class website.
• Usually, the slides will be available the evening before the lecture.
• The slides will be updated and reposted after class.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
3/37
Overview of Math 171A
• The lectures will be recorded and uploaded to Canvas “later in the day”.
• Lecture slides are available for download on Canvas and the class website.
• Usually, the slides will be available the evening before the lecture.
• The slides will be updated and reposted after class.
• If you print, please be mindful of the paper used – there are paper-friendly 4x4 versions of
the slides available
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
3/37
Overview of Math 171A
• The lectures will be recorded and uploaded to Canvas “later in the day”.
• Lecture slides are available for download on Canvas and the class website.
• Usually, the slides will be available the evening before the lecture.
• The slides will be updated and reposted after class.
• If you print, please be mindful of the paper used – there are paper-friendly 4x4 versions of
the slides available
• The login for the class website is the unique part of your UCSD email address,
e.g., if your email address is “pgill@ucsd.edu” then your account name is “pgill”.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
3/37
Overview of Math 171A
• The lectures will be recorded and uploaded to Canvas “later in the day”.
• Lecture slides are available for download on Canvas and the class website.
• Usually, the slides will be available the evening before the lecture.
• The slides will be updated and reposted after class.
• If you print, please be mindful of the paper used – there are paper-friendly 4x4 versions of
the slides available
• The login for the class website is the unique part of your UCSD email address,
e.g., if your email address is “pgill@ucsd.edu” then your account name is “pgill”.
• The password is your unique student ID in upper case, e.g., “A12345678 ”.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
3/37
Overview of Math 171A
The grade is based on homework, two midterms and a final.
Homework
20%
Midterm #1
(Friday Jan 29):
20%
Midterm #2
(Friday Feb 26):
20%
Final
(Friday, March 17, 11:30am–2:30pm):
40%
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
4/37
Overview of Math 171A
Homework
Homework assignments are posted each Monday, and are due 11:59pm on the Wednesday of the
following week (i.e., ten days after posting).
• Upload your solutions to Gradescope via Canvas.
• Some homework assignments require the use of Matlab.
Some programming experience recommended
(Matlab, Math 18/20D)
• In order to receive credit you must complete all the programming homework assignments.
• There will be no programming questions on the midterms or final.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
5/37
What is Optimization?
Webster’s dictionary:
Optimization “the process or method for making something (design, system, decision) as fully
perfect, functional or effective as possible.”
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
6/37
What is Optimization?
Webster’s dictionary:
Optimization “the process or method for making something (design, system, decision) as fully
perfect, functional or effective as possible.”
We often consciously or unconsciously optimize what we do, e.g., maximize savings, minimize
cost.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
6/37
What is Optimization?
Webster’s dictionary:
Optimization “the process or method for making something (design, system, decision) as fully
perfect, functional or effective as possible.”
We often consciously or unconsciously optimize what we do, e.g., maximize savings, minimize
cost.
The study of optimization involves the formulation of a mathematical model of a given process
or resource for which “optimizing” means minimizing or maximizing a function.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
6/37
Main Topics of the Class
The two steps of solving an optimization problem
(A) Model the problem
(B) Optimize the model
This class focuses on Step (B).
Introduction to algorithms for optimization.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
8/37
• The formulation of simple linear programs
— The graphical method
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
9/37
• The formulation of simple linear programs
— The graphical method
• Review of some basic linear algebra
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
9/37
• The formulation of simple linear programs
— The graphical method
• Review of some basic linear algebra
• Linear programs with constraints in all-inequality form and standard form
— optimality conditions
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
9/37
• The formulation of simple linear programs
— The graphical method
• Review of some basic linear algebra
• Linear programs with constraints in all-inequality form and standard form
— optimality conditions
• The simplex method for linear programming
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
9/37
• The formulation of simple linear programs
— The graphical method
• Review of some basic linear algebra
• Linear programs with constraints in all-inequality form and standard form
— optimality conditions
• The simplex method for linear programming
• Finding a feasible point
• Primal and dual forms of a linear program
• Interior methods
• Selected applications from engineering and data science
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
9/37
A Simple Modeling Example
The JuiceCo© company produces CranApple and AppleBerry juice. Both are made from a
blend of cranberry and apple juice.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
10/37
A Simple Modeling Example
The JuiceCo© company produces CranApple and AppleBerry juice. Both are made from a
blend of cranberry and apple juice.
Product
Cranberry
Apple
Profit
1 gallonCranApple =
3 quarts
1 quart
$3 per gallon
1 gallonAppleBerry =
2 quarts
2 quarts
$4 per gallon
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
10/37
A Simple Modeling Example
The JuiceCo© company produces CranApple and AppleBerry juice. Both are made from a
blend of cranberry and apple juice.
Product
Cranberry
Apple
Profit
1 gallonCranApple =
3 quarts
1 quart
$3 per gallon
1 gallonAppleBerry =
2 quarts
2 quarts
$4 per gallon
Suppose that there is a delivery of 75 gallons of juice:
200 quarts cranberry juice
100 quarts apple juice
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
10/37
A Simple Modeling Example
The JuiceCo© company produces CranApple and AppleBerry juice. Both are made from a
blend of cranberry and apple juice.
Product
Cranberry
Apple
Profit
1 gallonCranApple =
3 quarts
1 quart
$3 per gallon
1 gallonAppleBerry =
2 quarts
2 quarts
$4 per gallon
Suppose that there is a delivery of 75 gallons of juice:
200 quarts cranberry juice
100 quarts apple juice
How many gallons of CranApple and AppleBerry juice should be made to maximize profit?
(Assuming that all juice is sold.)
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
10/37
Variables
Step 1: Identify the variables
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
11/37
Variables
Step 1: Identify the variables
x = gallons of cranapple juice produced
y = gallons of appleberry juice produced
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
11/37
Variables
Step 1: Identify the variables
x = gallons of cranapple juice produced
y = gallons of appleberry juice produced
The problem has 2 variables. We want to find the “best” or “optimal” values for x and y
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
11/37
Objective
Step 2: Identify the objective
Find x and y to maximize profit.
CranApple sells for a profit of $3 per gallon and AppleBerry for $4 per gallon.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
12/37
Objective
Step 2: Identify the objective
Find x and y to maximize profit.
CranApple sells for a profit of $3 per gallon and AppleBerry for $4 per gallon.
The objective function is 3x + 4y .
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
12/37
Constraints
Step 3: Identify the constraints
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
13/37
Constraints
Step 3: Identify the constraints
What are some of the limitations on JuiceCo during production?
• JuiceCo has 200 quarts of cranberry juice and 100 quarts of apple juice.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
13/37
Constraints
Step 3: Identify the constraints
What are some of the limitations on JuiceCo during production?
• JuiceCo has 200 quarts of cranberry juice and 100 quarts of apple juice.
• 1 gallon of CranApple requires 3 quarts of cranberry juice and 1 quart of apple juice.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
13/37
Constraints
Step 3: Identify the constraints
What are some of the limitations on JuiceCo during production?
• JuiceCo has 200 quarts of cranberry juice and 100 quarts of apple juice.
• 1 gallon of CranApple requires 3 quarts of cranberry juice and 1 quart of apple juice.
• 1 gallon of AppleBerry requires 2 quarts of cranberry and 2 quarts of apple juice.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
13/37
Constraints
Step 3: Identify the constraints
What are some of the limitations on JuiceCo during production?
• JuiceCo has 200 quarts of cranberry juice and 100 quarts of apple juice.
• 1 gallon of CranApple requires 3 quarts of cranberry juice and 1 quart of apple juice.
• 1 gallon of AppleBerry requires 2 quarts of cranberry and 2 quarts of apple juice.
(cranberry juice used)
(apple
juice used)
3x + 2y ≤ 200
x + 2y ≤ 100
(total cranberry juice available)
(total apple
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
juice available).
13/37
Constraints
Step 3: Identify the constraints
What are some of the limitations on JuiceCo during production?
• JuiceCo has 200 quarts of cranberry juice and 100 quarts of apple juice.
• 1 gallon of CranApple requires 3 quarts of cranberry juice and 1 quart of apple juice.
• 1 gallon of AppleBerry requires 2 quarts of cranberry and 2 quarts of apple juice.
(cranberry juice used)
(apple
juice used)
3x + 2y ≤ 200
x + 2y ≤ 100
(total cranberry juice available)
(total apple
juice available).
Also, we can’t produce negative amounts of cranapple or appleberry juice!
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
⇒ x, y ≥ 0.
13/37
Mathematical problem
maximize
x,y
3x + 4y
subject to 3x + 2y ≤ 200
x + 2y ≤ 100
x
≥
0
y ≥
0.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
14/37
Mathematical problem
maximize
x,y
3x + 4y
subject to 3x + 2y ≤ 200
x + 2y ≤ 100
x
≥
0
y ≥
0.
Notice, the objective function and the constraints are all linear .
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
14/37
Maximizing profit is the same as minimizing loss:
maximize 3x + 4y
x,y
is the same as
minimize
x,y
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
− 3x − 4y .
15/37
Maximizing profit is the same as minimizing loss:
maximize 3x + 4y
x,y
is the same as
minimize
x,y
− 3x − 4y .
The juice problem may be written as the minimization problem:
minimize
x,y
subject to
− 3x − 4y
3x + 2y ≤ 200
x + 2y ≤ 100
x
≥
0
y ≥
0.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
15/37
The problems to be considered
In general we have “constraint functions”
a(x, y , z, . . . ) ≥ 0
b(x, y , z, . . . ) ≥ 0
..
..
.
.
e.g., the cranberry constraint is a(x, y ) = 200 − 3x − 2y ≥ 0.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
16/37
The problems to be considered
In general we have “constraint functions”
a(x, y , z, . . . ) ≥ 0
b(x, y , z, . . . ) ≥ 0
..
..
.
.
e.g., the cranberry constraint is a(x, y ) = 200 − 3x − 2y ≥ 0.
In general, the constraint functions may be nonlinear .
For example, x and y may need to be inside a circle, i.e.,
1 − x 2 − y 2 ≥ 0.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
16/37
Some examples of optimization problems
Data science:
• Compressive sensing: minimizing combinations of norms.
• Primal and dual forms of linear support-vector machines.
• Deep learning.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
17/37
Some examples of optimization problems
Data science:
• Compressive sensing: minimizing combinations of norms.
• Primal and dual forms of linear support-vector machines.
• Deep learning.
Inverse problems:
• Medical imaging
• Geophysical applications
— Estimating the Earth’s electrical conductivity
— Mineral and oil exploration
• Controlling the environment
— Cleaning groundwater contaminants
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
17/37
Air travel:
• Ticket payment
— An optimization algorithm checks your credit card transaction
• Flight attendants
— A flight scheduling algorithm assigned the flight crew to your plane
• Aircraft design
— An optimization algorithm was used to design shape of the plane
• Passenger routing
— NBA game locations and dates are determined using optimization
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
18/37
Design technology:
• Yacht design
— Optimal design of America’s Cup boats
• Space-X Falcon 9
— The take-off and re-entry trajectories calculated to minimize structural stress and
heating
— Determination of the surface temperature using sensors
— Minimization of fuel consumption and maximization of payload
• Calculation of trajectories for spacecraft
— Optimal control of long-term interplanetary missions
— Design of orbiters
• Control of unmanned autonomous vehicles (UAVs)
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
19/37
Structure of the class
Math 171A: minimizing linear functions subject to linear constraints
Linear programming
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
20/37
Structure of the class
Math 171A: minimizing linear functions subject to linear constraints
Linear programming
Math 171B: minimizing nonlinear functions subject to nonlinear constraints
Nonlinear programming
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
20/37
Examples of linear programs
• Mixture problems (in class)
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
21/37
Examples of linear programs
• Mixture problems (in class)
• Diet problems (in the Homework)
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
21/37
Examples of linear programs
• Mixture problems (in class)
• Diet problems (in the Homework)
• Financial optimization (in the text)
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
21/37
Can we use methods from calculus?
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
22/37
Can we use methods from calculus?
f (x) = x2
For example:
minimize x 2
x
The objective function f (x) = x 2 is minimized at x ∗ = 0.
The derivative is zero at x ∗
df
= f 0 (x) = 2x = 0.
dx
−2
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
−1
0
+1
+2
22/37
Can we use methods from calculus?
Similarly, f (x, y ) = x 2 + y 2 is minimized at x ∗ = 0,
y ∗ = 0.
The partial derivatives are zero at x ∗ and y ∗
∂f
= 2x = 0.
∂x
∂f
= 2y = 0.
∂y
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
x
y
23/37
The juice problem
Our problem is more difficult because of the inequality constraints.
UC San Diego | Center for Computational Mathematics | Math 171A: Linear Programming Winter 2021
24/37
The juice problem
Our problem is more difficult because of the inequality constraints.
maximize
x,y
3x + 4y
(objective function)
subject to 3x + 2y ≤ 200 (cranberry juice constraint)
x + 2y ≤ 100 (apple juice constraint)
x
≥
0 (minimum resource constraint)
y ...