Pre-(r)amble

The odds favor that by the time someone has reached this article, myself included, they have spent at least the briefest of moments (frustratedly?) questioning the practical applications for  linear combination, linear independence and linear math. In a sentence, these concepts allow us to mathematically understand and represent multidimensional coordinate systems. If you’re looking for a quick explanation for a homework problem feel free to skim through the bolded topics for help in specific areas of concern. Otherwise, here’s something to think about.  Imagine maneuvering in three dimensional space. An instantaneous position can be described using a three dimensional coordinate system. When following a consistent pattern of movement, an instantaneous position can be described with a fourth dimension, time. Suppose you have just landed the snowball throw of a lifetime and hit a target moving across your view plane, increasing the distance between you, and uphill. You have properly estimated the intersection of two moving objects in four dimensions. This is not always an easy task to execute. Now make this throw using a fifth dimension. Most people can’t comprehend the existence of a fifth dimension without having to understand how to maneuver in it. With linear math we can attempt to understand and represent the relationships between these dimensions.

Important Definitions

Linear Independence
A set of linearly independent vectors {} has ONLY the zero (trivial) solution <> <>  for the equation

Linear Dependence
Alternatively, if or , the set of vectors is said to be linearly dependent.

Determining Linear Independence

By row reducing a coefficient matrix created from our vectors {}, we can determine our <>. Then to classify a set of vectors as linearly independent or dependent, we compare to the definitions above.

Example
Determine if the following set of vectors are linearly independent:

, , ,

Setting up a Corresponding System of Equations and Finding it’s RREF Matrix

We need to understand that our vectors can be represented with a system of equations all equaling zero to satisfy the equation from our definition of linear independence. These equations will look something like this:

Notice that I have simply taken the coefficients from the given vectors and multiplied them by four variables (the number of variables will equal the number of vectors in the given set). They have been set equal to zero to allow us to test for linear independence. From here, create a coefficient matrix and perform row operations to reduce the matrix to reduced row echelon form (rref) .

rref =

Finding the Solution of the RREF Matrix

Finding the solution of the rref matrix may be the more difficult step in this process. However, it may become trivial following a few simple steps.

1) Identify the free variables in the matrix. Free variables are non-zero and located to the right of pivot variables. Pivot variables are the first non-zero entry in each row and since we have taken the rref of our matrix, all of the pivot variable coefficients are 1. By locating all free variables (or by eliminating all pivot variables) we find that is our only free variable.

2) Write free variables into your solution. The variable can be written into our solution vector as itself but we will represent it with another variable name (i.e. ) so that our solution is in parametric form. Multiple free variables are represented with multiple variables names (i.e. ). After this step your solution vector should look like this: <> <>.

3) Solve for pivot variables. The pivot variables should either be constant (i.e. 0, 6) or a function of your free variables (i.e. ). From the rref matrix we can see that , , and .

4) Complete the solution vector. Placing the values we just calculated into our solution vector: <> <>

Finally,

Since not all of our , the given set of vectors is said to be linearly dependent. The linear dependence relation is written using our solution vector multiplied by the respective vector from the given set: . We can also conclude that any vectors with non-zero coefficients are linear combinations of each other. Therefore, and are a linear combination.

Introduction to the convolution

Amongst the concepts that cause the most confusion to electrical engineering students, the Convolution Integral stands as a repeat offender.  As such, the point of this article is to explain what a convolution integral is, why engineers need it, and the math behind it.

In essence, the “convolution” of two functions (over the same variable, e.g. and ) is an operation that produces a separate third function that describes how the first function “modifies” the second one.  Conversely, the resulting function can be seen as how the second function “modifies” the first function.  Sometimes the result is used to describe how much the first two functions “have in common.”  In all honesty, the concept of the convolution of two functions is quite abstract, but the frequency at which it appears in nature grants its importance to scientists and engineers.  Ultimately the aim here is to identify its use to electrical engineers – so for now do not dwell solely on its mathematical significance.

A convolution of two functions is denoted with the operator “”, and is written as:

Convolution of f1(t) and f2(t)

Where is used as a “dummy variable.”  To aid in understanding this equation, observe the following graphic:

Convolution of two square pulses, resulting in a triangular pulse

Before diving any further into the math, let us first discuss the relevance of this equation to the realm of electrical engineering.

Why is the convolution integral relevant?

Most electrical circuits are designed to be linear, time-invariant (LTI) systems.  Being “linear” implies that the magnitude of a circuit’s output signal is a scaled version of the input signal’s magnitude.  Further, an LTI system that is excited by two independent signal sources will output the sum of the scaled versions of each signal.  This is extended for an infinite number of independent signal sources, and gives rise to the concept of superposition.  Put in another way, if a function causes an LTI system to output , then:

Where is a multiplicative constant.  In addition to this, superposition allows us to say:

Being a “time-invariant” system means it does not matter when the input signal is applied – a specific input signal will always result in the same output signal for a given LTI system.  Put mathematically, time-invariance can be expressed as:

where can be viewed as a time delay when dealing with signals through time (i.e. “time-domain signals”).  Though not directly, this concept also signifies that an output signal cannot contain frequency components not inherent in the input signal (causality).

The vast majority of circuits are LTI systems, each with a specific impulse response. The “impulse response” of a system is a system’s output when its input is fed with an impulse signal – a signal of infinitesimally short duration.  A real-world “impulse signal” would be something like a lightning bolt – or any form of ESD (electro-static dischage).   Basically, any voltage or current that spikes in magnitude for a relatively short period of time may be viewed as an impulse signal.  The impulse response of a circuit will always be a time-domain signal, and exists because no signal can propagate through a circuit in zero time; each individual electron involved can only move so quickly through each component.  Typically, real-world electronic LTI systems exhibit an impulse response that consists of an initial spike in magnitude, followed by an everlasting and ever-decreasing exponential relationship in signal magnitude.  The following image describes this graphically.

Typical Unit Impulse Response

So, here’s the big deal: the fact that each LTI circuit has a specific impulse response function (here, referred to as ) is very useful in predicting its behavior given a particular input signal (here, referred to as ).  This is because the input signal itself may be viewed as an impulse train – a stream of continuous impulse functions, with infinitesimally short durations of time between each impulse.  This fact, along with superposition, allows one to find the output of an LTI system given an arbitrary input signal by summing the LTI system’s impulse response to each impulse function that make up the input signal. By allowing the time between each “impulse” of the input signal to go to zero, this approach can be used to determine the output time-domain signal of an LTI system for any time-domain input signal.  For example, the following graphic shows the output of an RC circuit when fed with a square pulse:

Convolution of RC network impulse response and square wave input to find the output signal.

What is seen here is the integral of the impulse response and the input square wave as the square wave is stepped through time. In the above convolution equation, it is seen that the operation is done with respect to , a dummy variable.  In reality, we are taking an input signal, flipping it vertically through the origin (not evident with a square wave), and determining what the integral is at each value of , which here is delay through time. Since the output of any LTI system is non-causal (meaning it cannot exist until the signal that excites the output has been applied), we must mathematically step through time to see how each impulse signal of the input affects the LTI system’s impulse response – again, achieved by stepping through – the “time-delay” dummy variable.

A Convolution Example

To see how the convolution integral can be used to predict the output of an LTI circuit, observe the following example:

For an LTI system with an impulse response of , calculate the output, , given the input of:

The output of this system is found by solving:

We only integrate between 0 and + because, if we define as the time that the input signal is applied, then both and have zero magnitude at any time .

From there, we calculate:

Next, we can simplify and compute the integral:

Since for all , we can write the output as:

This result describes the output function for an LTI system with an impulse response when fed the input signal .

5 Steps to perform mathematical convolution

Often, one may wish to compute the convolution of two signals that can’t be described with one function of time alone.  For arbitrary signals, such as pulse trains or PCM signals, the convolution at any time t can be computed graphically.  For signals whose individual “sections” can be described mathematically, follow these steps to perform a convolution:

1.) Choose one of the two funtions ( or ), and leave it fixed in -space.

2.) Flip the other function vertically across the origin, so that it is time-inverted.

3.) Shift the inverted signal through the axis by seconds.  Choose to shift the signal to the first “section” of the fixed function that is described by the same equation.  The inverted signal (say, ), now shifted, represents , which is basically a “freeze frame” of the output after the input signal has been fed to the LTI system for seconds.

4.) The integral of the two functions, after shifting the inverted function by seconds, is the value of the convolution integral (i.e. output signal) at .

5.) Repeat this procedure through all “sections” of the function fixed in -space.  By doing this, you can compute the value of the output at any time !

Useful Properties

The following is a list of useful properties of the convolution integral that can help in developing an intuitive approach to solving problems:

1.) Commutative Property:

2.) Distributive Property:

3.) Associative Property:

4.) Shift Property:

if

then

5.) Convolution with an Impulse results in the original function:

where is the unit impulse function

6.) Width Property:

The convolution of a signal of duration and a signal of duration will result in a signal of duration

Convolution Table

Finally, here is a Convolution Table that can greatly reduce the difficulty in solving convolution integrals.

Thank you so much to Safa Khamis @ Kansas State University for taking the time to write this tutorial for Engineersphere and the electrical engineering community.

Matrix manipulations and properties

Finding the inverse of a matrix is much more complex than finding the inverse of a number. All real numbers have an inverse (i.e. ). However, not all matrices have an inverse. There are several characteristics that allow us to visibly determine whether a matrix has an inverse but we will only focus on one. A matrix must be square (i.e. 2×2, 3×3, etc.) to have an inverse. Performing the following manipulations will be a waste of time if a matrix is not square. It is also important to know the inverse matrix property. Using my example above, and similarly with matrices, where In is the identity matrix (diagonal from top left to bottom right contains all 1′s, and everything else is 0) . We take advantage of this property when solving systems of matrices.

In words, the general algorithm for determining the existence of an inverse matrix is to manipulate the matrix into row reduced echelon form (rref). If the rref matrix is an identity matrix, then the inverse matrix exists. Hang on now, earlier I mentioned that there were other, visible characteristics that allow us to determine the existence of an inverse matrix, but now I’m asking you to perform a tedious process (without a calculator) with the same goal? Wouldn’t it be easier to first determine if finding the rref of the matrix is worthwhile? You’re right, except we are going to make a simple manipulation, and at the same time that we finish our rref process and determine that an inverse matrix exists, we will have found the inverse matrix! How do we do that? We will create an augmented matrix between our matrix in question, , and the appropriate identity matrix where the size of matrix is equal to the size of matrix . We will perform the same rref process to the augmented matrix . If the portion of our augmented matrix previously belonging to matrix reduces to an identity matrix (indicating the existence of ), then the portion previously belonging to the identity matrix, will equal .

Some matrix math

Now, for the math…

Suppose we are asked to find the inverse of the following matrix:

First, we must set up the augmented matrix discussed above. Notice that I have simply placed the identity matrix (of the same size as ) on the right of matrix .

Finding the rref of an augmented matrix

Next, we will attempt to find the rref of the augmented matrix. If the portion of the augmented matrix previously belonging to yields an identity matrix, is invertible.

rref =

Ok great! The left half of our augmented matrix reduced to an identity matrix. That means two things to us: the matrix has an inverse and we’ve already found the inverse. If you recall from above, is the right half of the augmented matrix (after finding it’s rref, of course). So we can conclude:

If our rref of the augmented matrix had yielded anything other than an identity matrix, we would conclude that does not exist. This method will simply allow us to determine the existence of and entries to for any size matrix.

Describing the process of solving a linear system using the adjacent matrix is best done while performing an example. Suppose we have a system where is the coefficient matrix of our system, is the column vector containing our variables, and is the solution column vector. We are asked to solve for the column vector made up of variables , , and .

Typically, we would divide by to solve for , however there is no method for performing division between matrices. By taking advantage of the inverse matrix property , we can simply the formula to solve for the column vector . The commutative property does not apply in matrix multiplication so .  Therefore we have have to be aware of the ‘order’ in which we multiply:

      simplifies to     

Notice that since we multiplied by ‘first’ on the left side of the equation, we also multiply ‘first’ on the right side. Now, multiplying the inverse of matrix by matrix will yield a column vector matching our , , and . Below, I have used the equation and plugged the values for into the equation. The product between and is shown on the far right. Note: This article assumes you know how to find the inverse of a matrix. This process is described in my article Finding The Inverse of a Matrix.

Therefore, , , and . Simple systems (i.e. this 3×3 system) are much easier to solve with algebra instead of finding the inverse of the coefficient matrix and performing matrix multiplication. This application is more practical for larger systems or while working on Matrix Theory homework.

Please leave comments by signing in and then clicking on the “sticky note” located in the top right corner of this post to show your appreciation to the author!

We’ll begin with a square function, f(t), that has a an amplitude of 1, a start time of 2 seconds and an end time of 4 seconds.

Next, a time shift is demonstrated. Here our function is changed from f(t) to f(t-2). Notice that subtracting 2 from t in the function results in a positive shift of the graph.

In the next two graphs t will be scaled. Scaling t is not quite as intuitive as we may have expected. When we multiply t by 2, corresponding points of the function now occur at 1/2 the time they previously had. When we divide t by 2, each corresponding time on the graph occurs at a t that is now multiplied by 2. Notice that each of these factors directly affects the duration of the signal.

Scaling the amplitude has more intuitive results. If we multiply f(t) by 2, the amplitude of 1 is changed to 2. Multiplying f(t) by 1/2 results in an amplitude of 1/2.

Finally, multiplying t by -1 mirrors our function over the y-axis. Each time now occurs at its negative.

Example:
Here we will attempt to convert f(t) into 2*f(.5t+3). The graph of f(t) is shown below.

The easiest way to handle this type of problem without error is to manipulate the function one step at a time. First, I have converted f(t) into 2*f(t). Only the peaks are changed here (by a factor of 2).

Next, I convert 2*f(t) into 2*f( t). Notice how the actually expands our graph duration by a factor of 2 (from a 6 sec duration to a 12 sec duration).

Finally, we move from 2*f( t) to 2*f( t + 3). As shown in the discussion above, this is a time shift. Time shifts can be a little confusing because adding results in a negative shift of our graph. Try to think of it as our signal occurring 3 seconds earlier than before, reading from left to right on the graph. The easiest way to do this part is shift each x-intercept by 3 seconds (to the left, of course).

Why do we need to solve system equations?

Often during a course you will need to be able to solve a system equation for its roots.  These roots can be complex, distinct, or repeated.  These problems usually arise when working with linear systems or differential equations.  A system equation is formatted as follows:

System Equation:

How to solve a system equation

For example purposes, I will solve a system equation with complex roots.  A system equation with complex roots as a function of will appear in the following format (if it does not, you need to manipulate your equation to be in the form):

Roots:

So we have

which also equals

so your first step is to look at your equation and determine your roots, then write out your equation with constants.

Example with initial conditions and

so now we can write our equation as follows:

In order to solve for and we need to use our initial conditions.  To evaluate the first derivative initial condition, we must first take the derivative of our that we just found.

evaluating this equation with t = 0 and the response equal to -4v, we get this:

evaluating our equation with t = 0 and the response equal to 3v, we calculate

Using these two equations, we calculate our constants:

and

Fill these into our equation to determine the final result.

Now you know how to solve this common differential equations and linear systems problem, determine characteristic roots and modes, and write system equations.

What are indefinite integrals used for?

In integral can be thought of as an area underneath a curve.  Integrals are often used to manipulate position, velocity, and acceleration equations to estimate different situations.  If you are given an equation that represents the velocity of a golf cart driving, like so: (the ‘ in y’ represents the differential element that results when one performs a derivative on the position function f(x).)  then you can find use an integral (Anti-Derivative) to get an expression for the position of the golf cart, y(x).

The integral of a function is represented like so: and it can be thought of as a sum of areas like so:

Here the integral is performed on the function from point a to point b, which would make it a definite integral because the bounds are defined.  Our indefinite integral is the same procedure, except missing the bounds, which makes indefinite integral operation require a little twist.

We all know that the derivative of a constant is zero.  For instance, the derivative of 5 is equal to zero.  Once performing this derivative, we should still be able to perform the anti-derivative on this new function (zero) to obtain the original equation (5).  But how will we know what number permeates from performing an anti-derivative of zero.  An indefinite integral is called indefinite because the bounds are not defined on the integration, like so:  .  When we perform our indefinite integral we represent this long-lost constant by the letter ‘C’.

Before we integrate our golf cart velocity equation, lets go ahead and look at the laws of integration:

The integral can also be split up into separate individual integrals if there is addition in the function you are integrating.

and whenever you add these integrals together, you only need to account for 1 of the constants (C).

So the integral of our velocity will go as follows:

or

We can solve for our constant, C, if we are given initial conditions, such as the golf cart was moving at y’(0) = 1 m/s when we began collecting our data.  Otherwise, we leave the integral in this form.  If you would like to learn how to perform a definite integral, refer to our article on definite integrals.

The 3 common methods of describing a point in a three dimensional coordinate system are Cartesian, Cylindrical and Spherical. The most simple is Cartesian but certain teachers find it necessary to use the others. There are a few simple conversions between them but first it is necessary to know their notation.

Cartesian:

Cylindrical:

Spherical:

In most cases you will only need to work from Cartesian to Cylindrical or Spherical OR back, so I will only supply those equations. If you need to work between Cylindrical and Spherical, it would be one more simple step working from one of those, to Cartesian, then on to the other.

Cartesian Cylindrical:

Cartesian Cylindrical:

Cartesian Spherical:

Cartesian Spherical:

Plug the values from any given points into the correct equation to convert to a different type of coordinate system.

Suppose you are interested in finding the unit vector between two points, and , which are described in cartesian coordinates as and , respectively.

You would begin by finding the vector between these two points. The direction of this vector may be important so look for key words such as . Once we have established the direction we’re going in, to in this case, we subtract the beginning point from the end point. . This will give us the vector we are looking for. The next step would be to convert this vector into a unit vector, by dividing it by it’s magnitude.

These are the two formulas we are looking at:

Note:
Here I use as my vector from to and denotes the unit vector from to .

Implementing these formulas:

Hmm.. The End

Vector dot product rules

Another simple review of the vector dot product, for those of you that have forgotten.  The operation that involves multiplying two vectors together can be done in a few ways.  The first operation is called either the scalar product or the dot product.  One of the well known definitions looks like this:

RULE 1:

This is a scalar product that is equal to the two magnitudes multiplied together and multiplied by the cosine of the angle between them.  If  the two vectors are perpendicular to each other then the angle between them is 90 degrees, which will make the dot product equal zero.  This is an equivalent equation.

RULE 2:

When you finish your dot product, you should have a number, not a directional vector.  So if you get something like this you did something wrong: .  If you ended up with (any #) then you don’t have to completely rule out your answer.

Practice vectors

A few vectors to practice with:

Okay, now we have found our dot product by applying RULE 2 above.  We can use this value along with our individual vector magnitudes to apply RULE 1 and obtain the angle .

We can calculate the projection of the vector A onto the vector B by this relationship:

Note that this is a scalar quantity, and that we can also define the projection of B onto vector A in a similar fashion:

Performing a vector dot product in MATLAB

Perform a dot product in MATLAB like so:

>>A = [ 1 2 3];

>>B = [2 3 4];

>>dot(A,B)

Enjoy