Introduction to Simulation: Beginner


Linear Algebra

Solving Linear Equations
How to solve linear equations?
General form: Ax = b, where A is a matrix, x is a vector of variables you want to solve for, and b is a vector of constants.

So, the system of linear equations is when you have more than one variable and the same number of equations. One way of defining the problem is just writing the equations down explicitly like this or writing them down in a matrix form. So in this example,

A = [3 2 -1; 2 -2 4; -1 0.5 -1]
b = [1 -2 0]

A =
   3.0000   2.0000  -1.0000
   2.0000  -2.0000   4.0000
  -1.0000   0.5000  -1.0000

b =
   1  -2   0

When you are working with equations like this you might be tempted to do just simple algebra, because mathematically that works.
x = inv(A) * b'

x =
   1.0000
  -2.0000
  -2.0000

But if we try to do that:
inv(A)
inv(A) * b'

ans =
   2.2204e-16  -5.0000e-01  -2.0000e+00
   6.6667e-01   1.3333e+00   4.6667e+00
   3.3333e-01   1.1667e+00   3.3333e+00

ans =
   1.0000
  -2.0000
  -2.0000

Even though in the current example this approach worked, in many other cases MATLAB will give a warning because this is a quite inefficient way of calculating the answer. Instead, MATLAB already contained algorithms to solve such problems.

In the previous tutorial, we mentioned that there are different types of divisions: forward slash and backslash. The forward slash is the basic arithmetic operation, but the backslash is used exactly for solving the system of linear equation.
A\b'

ans =
   1.0000
  -2.0000
  -2.0000

Exercises

Now let's look at some other linear algebra concepts that we can do in MATLAB. For example, taking the determinant of the matrix. It's a simple function.

det(A)

ans = -3.0000

We can also compute eigenvalues of a matrix.
eig(A)

ans =
   3.6295
  -3.8445
   0.2150

And when we compute eigenvalues, we usually want the corresponding eigenvectors. This can be done through the same MATLAB function but a different function signature (check on the internet). So, let's try that.
[V, D] = eig(A)

V =
  -0.9601  -0.3061  -0.3288
  -0.2098   0.9134   0.7469
   0.1847  -0.2682   0.5780

D =
Diagonal Matrix
   3.6295        0        0
        0  -3.8445        0
        0        0   0.2150

We can verify the result. By the definition of eigenvalues and eigenvectors, A*V = V*D:
A*V == V*D

ans =
  0  0  0
  0  0  0
  0  0  0

In MATLAB, if we want to check if 2 things are equal, we can use == as we do in regular programming environments.
1 == 1
1 == 0

ans = 1
ans = 0

If we do A*V == V*D, we get false, which can be surprising because we expect them to be equal according to the definition of eigenvalues and eigenvectors. But here we have to remember that in programming there is always some round-off error. So, if we simply subtract A*V - V*D, we get back very small numbers.
A*V - V*D

ans =
   3.5527e-15   1.3323e-15   1.2906e-15
  -2.7756e-15  -1.3323e-15  -7.2164e-16
   1.1102e-16  -4.4409e-16   1.2490e-16

That brings us to the next topic to highlight. We have this matrix A*V - V*D that represents the differences in those 2 matrices, so we want to ensure that all values are small. There 9 different values in our example and there can be much bigger matrices than that, so sometimes it is really hard to tell how big or small are they collectively. In these cases, we want to calculate the distance from the origin. This can be done by calculating the norm of the vector of the matrix. In Euclidean space, it's all the elements squared, added together and then you take the square root. So, in MATLAB there is a built-in function, which gives us indeed a very small number.
norm(A*V - V*D)

ans = 5.0950e-15

If you remember from your math classes, there are different kinds of norms not only Euclidean, such as p-norm.


Back to Top