Random Processes
In practical problems, we deal with time varying waveforms whose value at a time is random in nature. For example, the speech waveform recorded by a microphone, the signal received by communication receiver or the daily record of stock-market data represents random variables that change with time. How do we characterize such data? Such data are characterized as random or stochastic processes. This lecture covers the fundamentals of random processes.
Random processes
Recall that a random variable maps each sample point in the sample space to a point in the real line. A random process maps each sample point to a waveform.
Consider a probability space 
. A random process can be defined on as an indexed family of random variables 
where 
is an index set, which may be discrete or continuous, usually denoting time. Thus a random process is a function of the sample point 
and index variable 
and may be written as 
.
Remark
-
For a fixed
is a random variable -
For a fixed 
, 
is a single realization of the random process and is a deterministic function.
For a fixed and a fixed is a single number.
When both and are varying we have the random process .
The random process
is normally denoted by 
Figure1 illustrates a random process.
Example 1 Consider a sinusoidal signal
where 
is a binary random variable with probability mass functions 
and 
Clearly,
is a random process with two possible realizations 
and 
At a particular time 
is a random variable with two values 
and 
.Continuous-time vs. Discrete-time process
If the index set
is continuous, 
is called a continuous-time process.Example 2 Suppose,
where and 
are constants and 
is uniformly distributed between0 and 
. 
is an example of a continuous-time process. -
If the index set
is a countable set, 
is called a discrete-time process. Such a random process can be represented as 
and called a random sequence. Sometimes the notation 
is used to describe a random sequence indexed by the set of positive integers.We can define a discrete-time random process on discrete points of time. Particularly, we can get a discrete-time random process
by sampling a continuous-time process \ at a uniform interval 
such that 
The discrete-time random process is more important in practical implementations. Advanced statistical signal processing techniques have been developed to process this type of signals.
Example 3 Suppose
where is a constant and 
is a random variable uniformly distributed between 
and 
.
Continuous-state vs. Discrete-state processThe value of a random process
is at any time can be described from its probabilistic model.The state is the value taken by
at a time , and the set of all such states is called the state space. A random process is discrete-state if the state-space is finite or countable. It also means that the corresponding sample space is also finite or countable. Otherwise , the random process is called continuous state.Example 4 Consider the random sequence
generated by repeated tossing of a fair coin where we assign 1 to Head and 0 to Tail.Clearly,
can take only two values - 0 and 1. Hence is a discrete-time two-state process. -
How to describe a random process?
As we have observed above that
at a specific time is a random variable and can be described by itsprobability distribution function 
This distribution function is called the first-order probability distribution function.
We can similarly define the first-order probability density function
To describe
, we have to use joint distribution function of the random variables at all possible values of . For any positive integer 
, 
represents jointly distributed random variables. Thus a random process can thus be described by specifying the 
joint distribution function .
or th the
joint probability density function
If
is a discrete-state random process, then it can be also specified by the collection of joint probability mass function
We can also define higher-order moments like
The above definitions are easily extended to a random sequence
.Examples of Random Processes
(a) Gaussian Random Process
For any positive integer
,
represent jointly random variables. These random variables define a random vector 
.The process 
is called Gaussian if the random vector 
is jointly Gaussian with the joint density function given by
where
and
