Multiple Constraints

Introduction

I have a previous post on Lagrange multipliers for a single constraint, however, as you have seen in some of my other posts. I have done constrained optimization on multiple constraints, in these cases I just add multiple Lagrange multipliers. So why do/can we do that?

Informal / Intuitive Proof

Consider we have a function $f:\mathbb{R}^n \rightarrow \mathbb{R}$, and $m$ constraint equations $g_i:\mathbb{R}^n \rightarrow \mathbb{R}$. Lets consider a point $x \in \mathbb{R}^n$ such that all the constraints are satisfied. Therefore, if x is a local min/max on this constraints set, if we inspect all the gradients, $\{\nabla_xg_i\}$ you will notice that they form a subspace and $\nabla_xf$ lies in this subspace which is why we can add more Lagrange multipliers since they are the coefficients for the linear combinations. But why does $\nabla_xf$ lie in the constraint gradients’ subspace? Well imagine it doesn’t, therefore it will have a component that is orthogonal to all the constraint gradients’. Because the gradient is normal to the tangent hyperplane of the contour set then a vector that is orthogonal to all the gradients, must lie in all the tangent planes. Therefore, that vector points in the direction of the intersection of all the constraints, therefore, $\nabla_xf$ can’t have any component along that direction since that will imply that $x$ is not a local min/max.