Multivariate Statistics Classical:Foundations and Modern Machine Learning.pdf.This book offers an in-depth examination of multivariate statistics, presenting both traditional and modern viewpoints. The first section addresses fundamental topics such as multivariate normality, MANOVA, discrimination, PCA, and canonical correlation analysis. The second section delves into contemporary concepts like gradient boosting, random forests, variable importance, and causal inference.
A key theme throughout the book is the application of classical multivariate statistics to elucidate advanced topics and to facilitate understanding of modern methods. For instance, linear models lay the groundwork for comprehending regularization through AIC and BIC, paving the way for a more in-depth analysis of regularization via generalization error and the VC theorem. Discriminant analysis introduces the weighted Bayes rule, which connects to modern classification methods for addressing class-imbalanced machine learning challenges. The method of steepest descent serves as a precursor to matching pursuit and gradient boosting, while axis-aligned trees like CART, a traditional tool, provide a foundation for newer techniques like super greedy trees.
Another significant theme is training error. Introductory courses often warn that overly aggressive reduction of training error can result in overfitting. Nevertheless, training error, also known as empirical risk, is a fundamental concept in statistical learning theory. In regression analysis, training error corresponds to the residual sum of squares; minimizing it yields the least squares solution, which can also lead to overfitting. Despite this concern, empirical risk is crucial for assessing the potential for effective learning. The principle of empirical risk minimization illustrates that minimizing training error can be beneficial, especially when combined with regularization. This concept is further explored through techniques such as penalization, matching pursuit, gradient boosting, and the construction of super greedy trees.
Key Features:
• Covers both classical and modern multivariate statistics.
• Each chapter includes a thoughtfully curated set of exercises that vary in difficulty and encompass both applied and theoretical aspects.
• The book serves as a valuable reference for researchers, featuring a wide range of topics, including new material on super greedy trees, rule-based variable selection, and machine learning for causal inference.
• Provides an extensive treatment of trees, offering a comprehensive and cohesive approach to understanding them in terms of partitions and empirical risk minimization.
• Includes new content on random forests, such as random forest quantile classifiers for class-imbalanced issues, multivariate random forests, subsampling methods for confidence regions, and super greedy forests. An entire chapter is dedicated to random survival forests, featuring fresh material on random hazard forests that extend survival forests to accommodate time-varying covariates.
