A fundamental problem that surfaces in all mathematical sciences is that of analyzing and solving algebraic equations in unknowns. The study of a system of simultaneous linear equations is in a natural and indivisible alliance with the study of the rectangular array of numbers defined by the coefficients of the equations.
Today, this problem would be formulated as three equations in three unknowns by writing
The coefficients and the constants are real or complex numbers. The unknowns are the variables of the system. The system is said to be linear if the functions are linear functions of the variables . The system is said to be homogeneous if the constants are all zero.
The system of equations can be written in matrix form as , where
In the above notation, the matrix is called the coefficient matrix, the vector is called the unknown vector, and the vector is called the constant vector. The system of equations is said to be consistent if there is at least one solution. The system of equations is said to be inconsistent if there is no solution. The system of equations is said to be dependent if there are infinitely many solutions.
As secondary students, we are familiar with the method of substitution and the method of elimination. The method of substitution is to solve one of the equations for one of the variables and substitute the result into the other equations. The method of elimination is to add or subtract multiples of the equations to eliminate one of the variables.
But in here, we will introduce the method of Gaussian elimination and the method of matrix inversion. The method of Gaussian elimination is similar to what we have learned in secondary school, but it is more systematic and efficient.
Before we introduce the method of Gaussian elimination, we need to first observe the linear system.
For such system, there are three possibilities:
Then we may introduce the idea of augmented matrix. The augmented matrix of the system is the matrix obtained by adjoining the column of constants to the coefficient matrix. For the above system, we may take the augmented matrix as .
If this augmented matrix has the form
then the system has a unique solution. We called this form as the row echelon form. The system has no solution if the row echelon form has a row of the form , where . The system has infinitely many solutions if the row echelon form has a row of the form .
Gaussian elimination is a methodical process of systematically transforming one system into another simpler, but equivalent, system (two systems are called equivalent if they possess equal solution sets) by successively eliminating unknowns and eventually arriving at a system that is easily solvable. The elimination process relies on three simple operations by which to transform one system to another equivalent system.
To describe these operations, let denote the $k$th equation:
and write the system of equations as
For a linear system , each of the following operations is called an elementary row operation:
The reason for applying these operations cannot change the solution set of the system is that each operation is reversible. For example, if we interchange two equations, we can interchange them back. If we multiply an equation by a nonzero constant, we can multiply it by the reciprocal of the constant. If we add a multiple of one equation to another, we can subtract the multiple of one equation from another.
The most common problem encounted in practice is the one which there are exactly equations and unknowns. In this case, we called the system as a square system. The method of Gaussian elimination is particularly efficient for solving square systems.
Take the following system as an example:
We may write the augmented matrix as
The first step is to eliminate the in the second and third equations. We may multiply the first equation by and subtract it from the second equation, and multiply the first equation by and add it to the third equation. The augmented matrix becomes
The second step is to eliminate the in the third equation. We may multiply the second equation by and add it to the third equation. The augmented matrix becomes
The third step is to eliminate the in the second equation. We may multiply the third equation by and add it to the second equation. The augmented matrix becomes
The fourth step is to time the second equation by and divide the third equation by , then we get the row echelon form
For simplicity, we may also eliminate the and in the first equation. The augmented matrix becomes
Then we may solve the system by back substitution. We have , , and . The solution is unique.
We may use the numpy library to solve the linear equation. The numpy.linalg.solve function can be used to solve the linear equation.
import numpy as np
A = np.array([[2, 1, 1], [6, 2, 1], [-2, 2, 1]]) # Coefficient matrix
b = np.array([1, -1, 7]) # Constants
x = np.linalg.solve(A, b) # Solve the linear equation
print(x) # Print the solution
You first need to write the augmented matrix and then use the method of Gaussian elimination to solve the system. Finally, use Python to verify your answer.
You first need to write the augmented matrix and then use the method of Gaussian elimination to solve the system. Finally, use Python to verify your answer.
The method of Gaussian elimination is to transform the augmented matrix into row echelon form. The method of Gauss-Jordan elimination is to transform the augmented matrix into reduced row echelon form. The reduced row echelon form is a form that is even simpler than the row echelon form.
The reduced row echelon form is a form that has the following properties:
The method of Gauss-Jordan elimination is similar to the method of Gaussian elimination. The only difference is that we need to make the leading entry in each row to be and make the leading in each row to be the only nonzero entry in its column.
Take the following system as an example:
For they are floating-point numbers, we better to use Python to solve the system. We may use the numpy.linalg.solve function to solve the system.
import numpy as np
A = np.array([[0.835, 0.667], [0.333, 0.266]]) # Coefficient matrix
b = np.array([0.168, 0.067]) # Constants
x = np.linalg.solve(A, b) # Solve the linear equation
print(x) # Print the solution
The solution is and . But if we perturb the constants a little bit, the solution may change. For example, if we change to , the solution may then change to and . This is called the ill-conditioned system.
This is an example of an ill-conditioned system. An ill-conditioned system is a system that is very sensitive to small changes in the constants. The condition number of a matrix is a measure of how sensitive the system is to small changes in the constants.
If and are two matrices of the same size, then the sum is the matrix obtained by adding the corresponding entries of and .
For example, if
then
If is a matrix of size and is a matrix of size , then the product is a matrix of size .
For example, if
then
However, one may note that
If is a square matrix, then we can form the product , which is a square matrix of the same size as . It is denoted as . Similarly, for any general matrix , we can form the product for any positive integer .
We define the unit matrix to be the matrix with 's on the diagonal and 's elsewhere. For example, the unit matrix is
In general, the unit matrix is
One may note that if , then is equivalent to . This is because for any vector . The matrix is called the identity matrix.
We can define for any square matrix . We can also define to be the matrix such that . The matrix is called the inverse of . If has an inverse, then is said to be invertible.
To find the inverse of a matrix, we may use the method of matrix inversion. The method of matrix inversion is to transform the augmented matrix into . The method of matrix inversion is similar to the method of Gaussian elimination.
For example, if
then we may write the augmented matrix as
By using the method of matrix inversion, we may transform the augmented matrix into
Therefore, the inverse of the matrix is
We may use the numpy library to find the inverse of a matrix. The numpy.linalg.inv function can be used to find the inverse of a matrix.
import numpy as np
A = np.array([[4, 7], [2, 6]]) # Coefficient matrix
A_inv = np.linalg.inv(A) # Find the inverse of the matrix
print(A_inv) # Print the inverse of the matrix
You first need to write the augmented matrix and then use the method of matrix inversion to find the inverse of the matrix. Finally, use Python to verify your answer.
You first need to write the augmented matrix and then use the method of matrix inversion to find the inverse of the matrix. Finally, use Python to verify your answer.
Recall what we have learned in the method of Gaussian elimination. The method of Gaussian elimination is to transform the augmented matrix into row echelon form. The row echelon form is the following:
for a matrix. Remember that when we talk about the possibility of the system, we have mentioned that the system has no solution if the row echelon form has a row of the form , where . To determine whether the system has solution, one may think of observing the leading in each row. If the leading in each row is the only nonzero entry in its column, then the system has a unique solution. If the leading in each row is not the only nonzero entry in its column, then the system has infinitely many solutions.
The rank of a matrix is the number of leading in the row echelon form. For example, the rank of the following matrix is .
However, the following matrix has rank .
One may wonder the reason why the rank of the matrix is . The reason is that the second row is a multiple of the first row. Therefore, the second row can be eliminated. The rank of the matrix is then .
Therefore, we may popose the following definition.
Definition(Linear Independence): A set of vectors is said to be linearly independent if the only solution to the equation is . Otherwise, the set of vectors is said to be linearly dependent.
To illustrate the concept of linear independence, we may consider the following example. The set of vectors and is linearly independent. This is because the only solution to the equation is . However, the set of vectors and is linearly dependent. This is because the equation has infinitely many solutions.
The rank of a matrix is the number of linearly independent rows in the matrix. The rank of a matrix is also the number of linearly independent columns in the matrix. The rank of a matrix is also the number of linearly independent vectors in the matrix.
while on the other hand, the nullity of a matrix is the number of free variables in the system. The nullity of a matrix is also the number of dependent rows in the matrix. The nullity of a matrix is also the number of dependent columns in the matrix. The nullity of a matrix is also the number of dependent vectors in the matrix.
For example, the following matrix has rank and nullity .
The following matrix has rank and nullity .
We may use the numpy library to find the rank and nullity of a matrix. The numpy.linalg.matrix_rank function can be used to find the rank of a matrix.
import numpy as np
A = np.array([[1, 2, 3], [2, 4, 6], [0, 0, 1]]) # Coefficient matrix
rank = np.linalg.matrix_rank(A) # Find the rank of the matrix
print(rank) # Print the rank of the matrix
The rank of the matrix is . The nullity of the matrix is then .
You first need to transform the matrix into row echelon form and then find the rank and nullity of the matrix. Finally, use Python to verify your answer.
The determinant of a square matrix is a scalar value that is a function of the entries of the matrix. The determinant of a matrix is denoted as or . The determinant of a matrix is
The determinant of a matrix is
For a more general matrix, the determinant can be found by expanding along any row or column. It is the the Laplace expansion, which is given by
where is the matrix obtained by deleting the $i$th row and $j$th column of .
Example: Find the determinant of the following matrix.
By applying the Laplace expansion along the first row, we have
We may use the numpy library to find the determinant of a matrix. The numpy.linalg.det function can be used to find the determinant of a matrix.
import numpy as np
A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) # Coefficient matrix
det = np.linalg.det(A) # Find the determinant of the matrix
print(det) # Print the determinant of the matrix
The determinant of the matrix is .
You first need to apply the Laplace expansion along the first row and then find the determinant of the matrix. Finally, use Python to verify your answer.