A system is memoryless, according to our definition, if $y(t) = f(x(t))$. If $y(t) = f(t,x(t))$, then the system has memory. Memoryless systems are \emph{instantaneous systems}

\begin{example}
Consider an AM Modulator defined by $y(t) = \cos(\omega_0 t)x(t)$. This system has memory because it must keep track of time to know what the value of $\cos(\omega_0 t)$ is. 
\end{example}

\begin{example}
Consider a resistor where the voltage across the circuit is given by $y(t) = Rx(t)$. This system is memoryless. If the voltage was given by $y(t) = R(t)x(t)$, then the system has memory.
\end{example}

If we consider an LTI system, what must be true of $h(n)$ for the system to be memoryless? Let's look at an output signal $y(n)$, whose value is the convolution of an arbitrary input $x(n)$ and $h(n)$.

\begin{align*}
y(n) &= \sum \limits_{k=0}^{\infty} h(n)x(k-n) \\
y(n) &= \cdots + h(-1)x(n+1) + h(0)x(n) + h(1)x(n-1) + \cdots \\
\end{align*}

Note that in order to have an instantaneous system without memory, the only term should be $h(0)x(n)$. So
$$y(n) = \sum \limits_{\ell=0}^{\infty} h(n)x(\ell-n) = kx(n)$$
where $k \in Z$. Therefore, $h(n) =k\delta(n)$ in order for the system to be memoryless.


Let $M$ be the statement ``the system is memoryless''.


Let $C$ be the statement ``the system is causal''. 


Is every memoryless system causal? In other words, is the following statement a tautology?
$$M \Rightarrow C$$
Since we define a memoryless system as a system whose output can be given by $y(t) = f(x(t))$, then we can prove this by using the definition of causality. Let $y_1(t) = f(x_1(t))$ be a memoryless system, and let $y_2(t) = f(x_2(t))$ be a memoryless system. It is clear that if $x_1(t) = x_2(t)$ $\forall t \leq T$, then $y_1(t) = y_2(t)$ $\forall t \leq T$, therefore, every memoryless system is causal, and the statement $M \Rightarrow C$ is a tautology.


Let $M$ be the statement ``the system is memoryless''.


Let $TI$ be the statement ``the system is time-invariant''.


Is every memoryless system time-invariant? In other words, is the following statement a tautology?
$$M \Rightarrow TI$$
We can also prove this with the definition of time-invariance. Let $y(t) = f(x(t))$, and let $\hat{x}(t) = x(t-T)$. Then we have $\hat{y}(t) = f(\hat{x}(t)) = f(x(t-T)) = y(t-T)$. Therefore, every memoryless system is time-invariant, and the statement $M \Rightarrow TI$ is a tautology.


Note that other textbooks say that this system is time-invariant: $y(t) = x(t)g(t)$. But note that $\hat{x}(t) = x(t-T) \Rightarrow \hat{y}(t) = \hat{x}(t)g(t) \neq y(t-T)$. Therefore, the system is not time-invariant. We can also deduce that the system is not memoryless since $\lnot TI \Rightarrow \lnot M$ is always true.


Given that a system is memoryless and $x(t_0) = x(t_1)$, we can deduce that $y(t_0) = y(t_1)$. In other words, $M \wedge \left( x(t_0)=x(t_1) \right) \Rightarrow y(t_0) = y(t_1)$.


To disprove memorylessness of a system, you can simply find a known memoryless signal and compare the signals where they have identical inputs at some time $t_0$. If $x_1(t_0) = x_2(t_0)$, then $y_1(t_0) \neq y_2(t_0) \Rightarrow$ the system has memory.  

Memory in Systems

BIBO stability stands for \emph{Bounded Input Bounded Output}. For a system that is BIBO stable, the response $y$ is bounded for every bounded input. The system $x \to \fbox{F} \to y$ is BIBO stable $\iff$ $\forall x \in X $ such that $x$ is bounded, $y$ is bounded such that $y$ is the response to $x$.

$$\mbox{The system } x \to \fbox{F} \to y \mbox{ is BIBO stable}$$
$$\iff$$
$$\abs{x(t)} \lt B_x \Rightarrow \exists B_y \gt 0 \mbox{ such that } \abs{y(t)} \l B_y \mbox{ }\forall t$$

\begin{example}
Is $y(n)=e^{\alpha x(n)}$ $\forall n $ for some $\alpha \in \R$ BIBO stable?

You can start with the assumption that $\abs{x(n)} \lt B_x$, then 
\begin{align*}
y(n) &= e^{\alpha x(n)} \\
\abs{y(n)} &= \abs{e^{\alpha x(n)}} \\
\abs{y(n)} &= \abs{e^{\alpha x(n)}} \lt \abs{e^{\alpha B_x}} = B_y  \\
\abs{y(n)} &\lt B_y
\end{align*}

Note that without loss of generality, $\alpha \gt 0$, since $-B_x \lt x(n) \lt B_x$ implies that $e^{\alpha B_x}$ and $e^{-\alpha B_x}$ are bounded the same. To see this you can look at the output values, $e^{-\alpha B_x}  \lt y(n) \lt e^{\alpha B_x}$, which can be simplified to $\abs{y(n)} \lt e^{\alpha B_x}$.
\end{example}

As long as you find a value that trumps in both the positive and negative range, then you have found a bound. The question will always involve whether or not you can find a bounded output given a bounded input.



BIBO Stability

Given an LTI system, the system is BIBO stable if and only if the impulse response is absolutely summable or absolutely integrable.

Let $LTI$ be the statement ``$H$ is an LTI system''. 

Let $BIBO$ be the statement ``$H$ is BIBO stable''.


Then the following statement is a tautology:
$$ LTI \wedge BIBO$$
$$ \iff $$
$$\lr{ \sum \limits_{n\in\Z}\abs{h(n)} \lt \infty}  \vee \lr{\int_{-\infty}^{\infty}\abs{h(t)}dt \lt \infty  }  $$

Here is the proof of the forward direction using summations:
\begin{proof}

First we are going to show $\sum \abs{h(n)} \lt \infty \Rightarrow BIBO$. Assume $\sum \limits_{n \in \Z} \abs{h(n)} \lt \infty$:
\begin{align*}
y(n) &= \sum \limits_{k\in\Z}h(k)x(n-k) \\
\abs{y(n)} &= \abs{\sum \limits_{k\in\Z}h(k)x(n-k)} \\
\abs{y(n)} &\leq \sum \limits_{k\in\Z}\abs{h(k)x(n-k)} \\
\abs{y(n)} &\leq \sum \limits_{k\in\Z}\abs{h(k)}\abs{x(n-k)} \\
\end{align*}

To make the input bounded, let $\abs{x(n)} \lt B_x$ $\forall n \in \Z$. Then

\begin{align*}
 \abs{y(n)} &\leq \sum \limits_{k\in\Z}\abs{h(k)}B_x \\
 \abs{y(n)} &\leq B_x\sum \limits_{k\in\Z}\abs{h(k)} \\
 \abs{y(n)} &\leq B_x\cdot k \quad \mbox{where } k = \sum \abs{h(n)} \lt \infty \\
 \abs{y(n)} &\leq B_y \\
\end{align*}

So given that an impulse response of a system is absolutely summable, when passed a bounded input signal, the system will always produce a bounded output signal. Therefore, any system whose impulse response is absolutely summable is BIBO stable.
\end{example}

\begin{example}
Given the system defined by $y(n) - \alpha y(n-1) = x(n)$, with zero initial conditions to make it an LTI system, for what values of $\alpha$ can this system be BIBO stable? \\

Solve this system in the forward direction $y(n) = \alpha y(n-1) + x(n)$:
\begin{align*}
y(n) &= x(n) + \alpha y(n-1) \\
y(n) &= x(n) + \alpha \lr{x(n-1) + \alpha y(n-2)} \\
y(n) &= x(n) + \alpha x(n-1) + \alpha^2 y(n-2) \\
y(n) &= x(n) + \alpha x(n-1) + \alpha^2\lr{x(n-2) + \alpha y(n-3)} \\
y(n) &= x(n) + \alpha x(n-1) + \alpha^2 x(n-2) + \alpha^3 y(n-3) \\
y(n) &= x(n) + \alpha x(n-1) + \alpha^2 x(n-2) + \alpha^3 x(n-3) + \cdots \\
y(n) &= \sum \limits_{k \in \Z_\oplus} \alpha^k x(n-k)
\end{align*}
This shows us that $h(n) = \begin{cases} 0 & n\lt0 \\ \alpha^n & n\geq 0 \end{cases} $, or $h(n) = \alpha^n u(n)$. Now we can determine which values for which the system is BIBO stable. We can look at the absolute sum of the impulse response $h(n)$:
$$ \sum \limits_{k \in \Z} \abs{h(n)} = \sum \limits_{k \in \Z_\oplus} \abs{\alpha^k} = \frac{1}{1-\abs{\alpha}} $$
But that last equality is true if and only if $\abs{\alpha} \lt 1$, thus our range of values for which the system is BIBO stable is $\alpha \in (-1,1)$.

\end{example}

It is important to note that you cannot assume that $H(\omega)$ exists to show that a system is BIBO stable, because that presumes BIBO stability.




So far, we showed that absolute summability or absolute integrability is a sufficient condition for BIBO stability. Lets show that it is also necessary. We want to show that BIBO stability implies absolute summability or absolute integrability, however, proving the contrapositive is much easier (we will show for absolute summability):

$$ \lnot \lr{\sum \limits_{n \in \Z} \abs{h(n)} \lt 0} \Rightarrow \lnot BIBO $$

Note the meaning of a system not being BIBO stable means that $\exists x \in X$ such that $\abs{x(n)} \lt B_x$ for some $B_x \lt \infty$ that produces an output signal that is unbounded. In other words, it produces a $y(n)$ such that $\abs{y(n)} = \infty$ for some $n$.

\begin{proof}

Let $h(n)$ be an impulse response to a system such that $\sum \abs{h(n)} = \infty$. Let $\hat{x}(n)$ be an input signal defined by

$$\hat{x}(n) = \begin{cases} \frac{h(n)}{\abs{h(n)}} & h(n) \neq 0 \\ 0 & \mbox{otherwise} \end{cases}$$

Let $x(n) = \hat{x}(-n)$.

$$x(n) = \begin{cases} \frac{h(-n)}{\abs{h(-n)}} & h(-n) \neq 0 \\ 0 & \mbox{otherwise} \end{cases}$$

Then we have our output signal defined by $y(n) = \sum_{k \in \Z}h(k)x(n-k)$. If we look at $y(0)$, we have
$$y(0) = \sum_{k \in Z}h(k)x(-k)$$
We can use this to show that we will have an unbounded output for any input. 
\begin{align*}
y(0) &= \sum \limits_{k \in Z}h(k)x(-k) \\
y(0) &= \sum \limits_{k \in Z}h(k)\frac{h(k)}{\abs{h(k)}} \\
y(0) &= \sum \limits_{k \in Z}\frac{\lr{h(k)}^2}{\abs{h(k)}} \\
y(0) &= \sum \limits_{k \in Z}\frac{\abs{h(k)}\abs{h(k)}}{\abs{h(k)}} \\
y(0) &= \sum \limits_{k \in Z}\abs{h(k)} = \infty \quad \mbox{(given)}\\
\end{align*}
Therefore, if a system's impulse response is not absolutely summable, the system cannot produce a bounded output when given a bounded input.
\end{proof}

What if $h(n):\Z \to \C$? Then we could have proved the same above by letting $\hat{x}(n) = \frac{h^*(n)}{\abs{h^*(n)}} = \frac{h^*(n)}{\abs{h(n)}}$. This will also "blow up". 

LTI Systems and BIBO Stability

Recall that the definition of a frequency response $H(\omega)$ is given by the summation (in DT):
$$ H(\omega) = \sum \limits_{n \in \Z}h(n)e^{-i\omega n}  $$

Notice that if we take the absolute value of the terms in the sum, we have
$$\sum \limits_{n \in \Z}\abs{h(n)e^{-i\omega n}} = \sum \limits_{n \in \Z}\abs{h(n)}  $$
Therefore, if $\sum \abs{h(n)}$ is divergent, then the frequency response has issues.
\begin{enumerate}
\item $\sum \abs{h(n)} \lt \infty \Rightarrow H(\omega) $ is nice and smooth.
\item $\sum \abs{h(n)} = \infty \wedge \sum \abs{h(n)}^2 \lt \infty \Rightarrow H(\omega) $ is bounded but will have discontinuities. Note that $\sum \abs{h(n)}^2$ represents the energy of the system, thus $\sum \abs{h(n)}^2 \lt \infty$ means that the system has finite energy.
\item $\sum \abs{h(n)} = \infty \wedge \sum \abs{h(n)}^2 = \infty \Rightarrow$ "all bets are off". Either $H(\omega)$ has impulses, or doesn't exist. The system may not have a frequency response.
\end{enumerate}

Let $BIBO$ be the statement that the system produces a bounded output for every bounded input.

Let $FE$ be the statement that the system has finite energy ($\sum \abs{h(n)}^2 \lt \infty$)

Let $F$ be the statement that the system has a frequency response.


Then we can sum up the above statements regarding frequency response:
\begin{align*}
BIBO &\Rightarrow F \\
\lnot BIBO \wedge FE &\Rightarrow F \\
\lnot BIBO \wedge \lnot FE &\Rightarrow F \vee \lnot F \\
\end{align*}


9 Week 9