Theory of Ridge Regression Estimation with Applications

Theory of Ridge Regression Estimation with Applications

by A. K. Md. Ehsanes Saleh, Mohammad Arashi, B. M. Golam Kibria
Released February 2019
Publisher(s): Wiley
ISBN: 9781118644614

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

A guide to the systematic analytical results for ridge, LASSO, preliminary test, and Stein-type estimators with applications

Theory of Ridge Regression Estimation with Applications offers a comprehensive guide to the theory and methods of estimation. Ridge regression and LASSO are at the center of all penalty estimators in a range of standard models that are used in many applied statistical analyses. Written by noted experts in the field, the book contains a thorough introduction to penalty and shrinkage estimation and explores the role that ridge, LASSO, and logistic regression play in the computer intensive area of neural network and big data analysis.

Designed to be accessible, the book presents detailed coverage of the basic terminology related to various models such as the location and simple linear models, normal and rank theory-based ridge, LASSO, preliminary test and Stein-type estimators.
The authors also include problem sets to enhance learning. This book is a volume in the Wiley Series in Probability and Statistics series that provides essential and invaluable reading for all statisticians. This important resource:

  • Offers theoretical coverage and computer-intensive applications of the procedures presented
  • Contains solutions and alternate methods for prediction accuracy and selecting model procedures
  • Presents the first book to focus on ridge regression and unifies past research with current methodology
  • Uses R throughout the text and includes a companion website containing convenient data sets

Written for graduate students, practitioners, and researchers in various fields of science, Theory of Ridge Regression Estimation with Applications is an authoritative guide to the theory and methodology of statistical estimation.

Table of contents

  1. Cover
  2. Dedication
  3. List of Figures
  4. List of Tables
  5. Preface
  6. Abbreviations and Acronyms
  7. List of Symbols
  8. 1 Introduction to Ridge Regression
    1. 1.1 Introduction
    2. 1.2 Ridge Regression Estimator: Ridge Notion
    3. 1.3 LSE vs. RRE
    4. 1.4 Estimation of Ridge Parameter
    5. 1.5 Preliminary Test and Stein‐Type Ridge Estimators
    6. 1.6 High‐Dimensional Setting
    7. 1.7 Notes and References
    8. 1.8 Organization of the Book
  9. 2 Location and Simple Linear Models
    1. 2.1 Introduction
    2. 2.2 Location Model
    3. 2.3 Simple Linear Model
    4. 2.4 Summary and Concluding Remarks
  10. 3 ANOVA Model
    1. 3.1 Introduction
    2. 3.2 Model, Estimation, and Tests
    3. 3.3 Bias and Weighted Risks of Estimators
    4. 3.4 Comparison of Estimators
    5. 3.5 Application
    6. 3.6 Efficiency in Terms of Unweighted Risk
    7. 3.7 Summary and Concluding Remarks
    8. 3.A Appendix
    9. Problems
  11. 4 Seemingly Unrelated Simple Linear Models
    1. 4.1 Model, Estimation, and Test of Hypothesis
    2. 4.2 Bias and MSE Expressions of the Estimators
    3. 4.3 Comparison of Estimators
    4. 4.4 Efficiency in Terms of Unweighted Risk
    5. 4.5 Summary and Concluding Remarks
  12. 5 Multiple Linear Regression Models
    1. 5.1 Introduction
    2. 5.2 Linear Model and the Estimators
    3. 5.3 Bias and Weighted Risks of Estimators
    4. 5.4 Comparison of Estimators
    5. 5.5 Efficiency in Terms of Unweighted Risk
    6. 5.6 Summary and Concluding Remarks
  13. 6 Ridge Regression in Theory and Applications
    1. 6.1 Multiple Linear Model Specification
    2. 6.2 Ridge Regression Estimators (RREs)
    3. 6.3 Bias, MSE, and Risk of Ridge Regression Estimator
    4. 6.4 Determination of the Tuning Parameters
    5. 6.5 Ridge Trace
    6. 6.6 Degrees of Freedom of RRE
    7. 6.7 Generalized Ridge Regression Estimators
    8. 6.8 LASSO and Adaptive Ridge Regression Estimators
    9. 6.9 Optimization Algorithm
    10. 6.10 Estimation of Regression Parameters for Low‐Dimensional Models
    11. 6.11 Summary and Concluding Remarks
  14. 7 Partially Linear Regression Models
    1. 7.1 Introduction
    2. 7.2 Partial Linear Model and Estimation
    3. 7.3 Ridge Estimators of Regression Parameter
    4. 7.4 Biases and Risks of Shrinkage Estimators
    5. 7.5 Numerical Analysis
    6. 7.6 High‐Dimensional PLM
    7. 7.7 Summary and Concluding Remarks
  15. 8 Logistic Regression Model
    1. 8.1 Introduction
    2. 8.2 Asymptotic Distributional Risk Efficiency Expressions of the Estimators
    3. 8.3 Summary and Concluding Remarks
  16. 9 Regression Models with Autoregressive Errors
    1. 9.1 Introduction
    2. 9.2 Asymptotic Distributional ‐risk Efficiency Comparison
    3. 9.3 Example: Sea Level Rise at Key West, Florida
    4. 9.4 Summary and Concluding Remarks
  17. 10 Rank‐Based Shrinkage Estimation
    1. 10.1 Introduction
    2. 10.2 Linear Model and Rank Estimation
    3. 10.3 Asymptotic Distributional Bias and Risk of the R‐Estimators
    4. 10.4 Comparison of Estimators
    5. 10.5 Summary and Concluding Remarks
  18. 11 High‐Dimensional Ridge Regression
    1. 11.1 High‐Dimensional RRE
    2. 11.2 High‐Dimensional Stein‐Type RRE
    3. 11.3 Post Selection Shrinkage
    4. 11.4 Summary and Concluding Remarks
  19. 12 Applications: Neural Networks and Big Data
    1. 12.1 Introduction
    2. 12.2 A Simple Two‐Layer Neural Network
    3. 12.3 Deep Neural Networks
    4. 12.4 Application: Image Recognition
    5. 12.5 Summary and Concluding Remarks
  20. References
  21. Index
  22. End User License Agreement

Product information

  • Title: Theory of Ridge Regression Estimation with Applications
  • Author(s): A. K. Md. Ehsanes Saleh, Mohammad Arashi, B. M. Golam Kibria
  • Release date: February 2019
  • Publisher(s): Wiley
  • ISBN: 9781118644614