Statistical Signal Processing in Engineering

Statistical Signal Processing in Engineering

by Umberto Spagnolini
Released February 2018
Publisher(s): Wiley
ISBN: 9781119293972

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

A problem-solving approach to statistical signal processing for practicing engineers, technicians, and graduate students 

This book takes a pragmatic approach in solving a set of common problems engineers and technicians encounter when processing signals. In writing it, the author drew on his vast theoretical and practical experience in the field to provide a quick-solution manual for technicians and engineers, offering field-tested solutions to most problems engineers can encounter. At the same time, the book delineates the basic concepts and applied mathematics underlying each solution so that readers can go deeper into the theory to gain a better idea of the solution’s limitations and potential pitfalls, and thus tailor the best solution for the specific engineering application. 

Uniquely, Statistical Signal Processing in Engineering can also function as a textbook for engineering graduates and post-graduates. Dr. Spagnolini, who has had a quarter of a century of experience teaching graduate-level courses in digital and statistical signal processing methods, provides a detailed axiomatic presentation of the conceptual and mathematical foundations of statistical signal processing that will challenge students’ analytical skills and motivate them to develop new applications on their own, or better understand the motivation underlining the existing solutions.  

Throughout the book, some real-world examples demonstrate how powerful a tool statistical signal processing is in practice across a wide range of applications.

  • Takes an interdisciplinary approach, integrating basic concepts and tools for statistical signal processing
  • Informed by its author’s vast experience as both a practitioner and teacher
  • Offers a hands-on approach to solving problems in statistical signal processing
  • Covers a broad range of applications, including communication systems, machine learning, wavefield and array processing, remote sensing, image filtering and distributed computations
  • Features numerous real-world examples from a wide range of applications showing the mathematical concepts involved in practice
  • Includes MATLAB code of many of the experiments in the book

Statistical Signal Processing in Engineering is an indispensable working resource for electrical engineers, especially those working in the information and communication technology (ICT) industry. It is also an ideal text for engineering students at large, applied mathematics post-graduates and advanced undergraduates in electrical engineering, applied statistics, and pure mathematics, studying statistical signal processing.

Table of contents

  1. Cover
  2. Title Page
  3. List of Figures
  4. List of Tables
  5. Preface
  6. List of Abbreviations
  7. How to Use the Book
  8. About the Companion Website
  9. Prerequisites
  10. Why are there so many matrixes in this book?
  11. 1 Manipulations on Matrixes
    1. 1.1 Matrix Properties
    2. 1.2 Eigen‐Decompositions
    3. 1.3 Eigenvectors in Everyday Life
    4. 1.4 Derivative Rules
    5. 1.5 Quadratic Forms
    6. 1.6 Diagonalization of a Quadratic Form
    7. 1.7 Rayleigh Quotient
    8. 1.8 Basics of Optimization
    9. Appendix A: Arithmetic vs. Geometric Mean
  12. 2 Linear Algebraic Systems
    1. 2.1 Problem Definition and Vector Spaces
    2. 2.2 Rotations
    3. 2.3 Projection Matrixes and Data‐Filtering
    4. 2.4 Singular Value Decomposition (SVD) and Subspaces
    5. 2.5 QR and Cholesky Factorization
    6. 2.6 Power Method for Leading Eigenvectors
    7. 2.7 Least Squares Solution of Overdetermined Linear Equations
    8. 2.8 Efficient Implementation of the LS Solution
    9. 2.9 Iterative Methods
  13. 3 Random Variables in Brief
    1. 3.1 Probability Density Function (pdf), Moments, and Other Useful Properties
    2. 3.2 Convexity and Jensen Inequality
    3. 3.3 Uncorrelatedness and Statistical Independence
    4. 3.4 Real‐Valued Gaussian Random Variables
    5. 3.5 Conditional pdf for Real‐Valued Gaussian Random Variables
    6. 3.6 Conditional pdf in Additive Noise Model
    7. 3.7 Complex Gaussian Random Variables
    8. 3.8 Sum of Square of Gaussians: Chi‐Square
    9. 3.9 Order Statistics for N rvs
  14. 4 Random Processes and Linear Systems
    1. 4.1 Moment Characterizations and Stationarity
    2. 4.2 Random Processes and Linear Systems
    3. 4.3 Complex‐Valued Random Processes
    4. 4.4 Pole‐Zero and Rational Spectra (Discrete‐Time)
    5. 4.5 Gaussian Random Process (Discrete‐Time)
    6. 4.6 Measuring Moments in Stochastic Processes
    7. Appendix A: Transforms for Continuous‐Time Signals
    8. Appendix B: Transforms for Discrete‐Time Signals
  15. 5 Models and Applications
    1. 5.1 Linear Regression Model
    2. 5.2 Linear Filtering Model
    3. 5.3 MIMO systems and Interference Models
    4. 5.4 Sinusoidal Signal
    5. 5.5 Irregular Sampling and Interpolation
    6. 5.6 Wavefield Sensing System
  16. 6 Estimation Theory
    1. 6.1 Historical Notes
    2. 6.2 Non‐Bayesian vs. Bayesian
    3. 6.3 Performance Metrics and Bounds
    4. 6.4 Statistics and Sufficient Statistics
    5. 6.5 MVU and BLU Estimators
    6. 6.6 BLUE for Linear Models
    7. 6.7 Example: BLUE of the Mean Value of Gaussian rvs
  17. 7 Parameter Estimation
    1. 7.1 Maximum Likelihood Estimation (MLE)
    2. 7.2 MLE for Gaussian Model
    3. 7.3 Other Noise Models
    4. 7.4 MLE and Nuisance Parameters
    5. 7.5 MLE for Continuous‐Time Signals
    6. 7.6 MLE for Circular Complex Gaussian
    7. 7.7 Estimation in Phase/Frequency Modulations
    8. 7.8 Least Squares (LS) Estimation
    9. 7.9 Robust Estimation
  18. 8 Cramér–Rao Bound
    1. 8.1 Cramér–Rao Bound and Fisher Information Matrix
    2. 8.2 Interpretation of CRB and Remarks
    3. 8.3 CRB and Variable Transformations
    4. 8.4 FIM for Gaussian Parametric Model
    5. Appendix A: Proof of CRB
    6. Appendix B: FIM for Gaussian Model
    7. Appendix C: Some Derivatives for MLE and CRB Computations
  19. 9 MLE and CRB for Some Selected Cases
    1. 9.1 Linear Regressions
    2. 9.2 Frequency Estimation
    3. 9.3 Estimation of Complex Sinusoid
    4. 9.4 Time of Delay Estimation
    5. 9.5 Estimation of Max for Uniform pdf
    6. 9.6 Estimation of Occurrence Probability for Binary pdf
    7. 9.7 How to Optimize Histograms?
    8. 9.8 Logistic Regression
  20. 10 Numerical Analysis and Montecarlo Simulations
    1. 10.1 System Identification and Channel Estimation
    2. 10.2 Frequency Estimation
    3. 10.3 Time of Delay Estimation
    4. 10.4 Doppler‐Radar System by Frequency Estimation
  21. 11 Bayesian Estimation
    1. 11.1 Additive Linear Model with Gaussian Noise
    2. 11.2 Bayesian Estimation in Gaussian Settings
    3. 11.3 LMMSE Estimation and Orthogonality
    4. 11.4 Bayesian CRB
    5. 11.5 Mixing Bayesian and Non‐Bayesian
    6. 11.6 Expectation‐Maximization (EM)
    7. Appendix Gaussian Mixture pdf
  22. 12 Optimal Filtering
    1. 12.1 Wiener Filter
    2. 12.2 MMSE Deconvolution (or Equalization)
    3. 12.3 Linear Prediction
    4. 12.4 LS Linear Prediction
    5. 12.5 Linear Prediction and AR Processes
    6. 12.6 Levinson Recursion and Lattice Predictors
  23. 13 Bayesian Tracking and Kalman Filter
    1. 13.1 Bayesian Tracking of State in Dynamic Systems
    2. 13.2 Kalman Filter (KF)
    3. 13.3 Identification of Time‐Varying Filters in Wireless Communication
    4. 13.4 Extended Kalman Filter (EKF) for Non‐Linear Dynamic Systems
    5. 13.5 Position Tracking by Multi‐Lateration
    6. 13.6 Non‐Gaussian Pdf and Particle Filters
  24. 14 Spectral Analysis
    1. 14.1 Periodogram
    2. 14.2 Parametric Spectral Analysis
    3. 14.3 AR Spectral Analysis
    4. 14.4 MA Spectral Analysis
    5. 14.5 ARMA Spectral Analysis
    6. Appendix A: Which Sample Estimate of the Autocorrelation to Use?
    7. Appendix B: Eigenvectors and Eigenvalues of Correlation Matrix
    8. Appendix C: Property of Monic Polynomial
    9. Appendix D: Variance of Pole in AR(1)
  25. 15 Adaptive Filtering
    1. 15.1 Adaptive Interference Cancellation
    2. 15.2 Adaptive Equalization in Communication Systems
    3. 15.3 Steepest Descent MSE Minimization
    4. 15.4 From Iterative to Adaptive Filters
    5. 15.5 LMS Algorithm and Stochastic Gradient
    6. 15.6 Convergence Analysis of LMS Algorithm
    7. 15.7 Learning Curve of LMS
    8. 15.8 NLMS Updating and Non‐Stationarity
    9. 15.9 Numerical Example: Adaptive Identification
    10. 15.10 RLS Algorithm
    11. 15.11 Exponentially‐Weighted RLS
    12. 15.12 LMS vs. RLS
    13. Appendix A: Convergence in Mean Square
  26. 16 Line Spectrum Analysis
    1. Why Line Spectrum Analysis?
    2. 16.1 Model Definition
    3. 16.2 Maximum Likelihood and Cramér–Rao Bounds
    4. 16.3 High‐Resolution Methods
  27. 17 Equalization in Communication Engineering
    1. 17.1 Linear Equalization
    2. 17.2 Non‐Linear Equalization
    3. 17.3 MIMO Linear Equalization
    4. 17.4 MIMO–DFE Equalization
  28. 18 2D Signals and Physical Filters
    1. 18.1 2D Sinusoids
    2. 18.2 2D Filtering
    3. 18.3 Diffusion Filtering
    4. 18.4 Laplace Equation and Exponential Filtering
    5. 18.5 Wavefield Propagation
    6. Appendix A: Properties of 2D Signals
    7. Appendix B: Properties of 2D Fourier Transform
    8. Appendix C: Finite Difference Method for PDE‐Diffusion
  29. 19 Array Processing
    1. 19.1 Narrowband Model
    2. 19.2 Beamforming and Signal Estimation
    3. 19.3 DoA Estimation
  30. 20 Multichannel Time of Delay Estimation
    1. 20.1 Model Definition for ToD
    2. 20.2 High Resolution Method for ToD (L = 1)
    3. 20.3 Difference of ToD (DToD) Estimation
    4. 20.4 Numerical Performance Analysis of DToD
    5. 20.5 Wavefront Estimation: Non‐Parametric Method (L = 1)
    6. 20.6 Parametric ToD Estimation and Wideband Beamforming
    7. Appendix A: Properties of the Sample Correlations
    8. Appendix B: How to Delay a Discrete‐Time Signal?
    9. Appendix C: Wavefront Estimation for 2D Arrays
  31. 21 Tomography
    1. 21.1 X‐ray Tomography
    2. 21.2 Algebraic Reconstruction Tomography (ART)
    3. 21.3 Reconstruction From Projections: Fourier Method
    4. 21.4 Traveltime Tomography
    5. 21.5 Internet (Network) Tomography
  32. 22 Cooperative Estimation
    1. 22.1 Consensus and Cooperation
    2. 22.2 Distributed Estimation for Arbitrary Linear Models (p>1)
    3. 22.3 Distributed Synchronization
    4. Appendix Basics of Undirected Graphs
  33. 23 Classification and Clustering
    1. 23.1 Historical Notes
    2. 23.2 Classification
    3. 23.3 Classification of Signals in Additive Gaussian Noise
    4. 23.4 Bayesian Classification
    5. 23.5 Pattern Recognition and Machine Learning
    6. 23.6 Clustering
  34. References
  35. Index
  36. End User License Agreement

Product information

  • Title: Statistical Signal Processing in Engineering
  • Author(s): Umberto Spagnolini
  • Release date: February 2018
  • Publisher(s): Wiley
  • ISBN: 9781119293972