Alppaydin Machine Learning 2010

Springer Series in Statistics, 2001

A linear regression model assumes that the regression function E(Y |X) is linear in the inputs X 1 ,. .. , X p. Linear models were largely developed in the precomputer age of statistics, but even in today's computer era there are still good reasons to study and use them. They are simple and often provide an adequate and interpretable description of how the inputs affect the output. For prediction purposes they can sometimes outperform fancier nonlinear models, especially in situations with small numbers of training cases, low signal-to-noise ratio or sparse data. Finally, linear methods can be applied to transformations of the inputs and this considerably expands their scope. These generalizations are sometimes called basis-function methods, and are discussed in Chapter 5. In this chapter we describe linear methods for regression, while in the next chapter we discuss linear methods for classification. On some topics we go into considerable detail, as it is our firm belief that an understanding of linear methods is essential for understanding nonlinear ones. In fact, many nonlinear techniques are direct generalizations of the linear methods discussed here.

Biometrics, 2009

Semi-Markov Chains and Hidden Semi-Markov Models toward Applications: Their Use in Reliability and DNA Analysis (V. S. Barbu and N. Limnios) Yann Guédon Brief Reports by the Editor Handbook of Statistical Analyses Using SAS, 3rd edition (G. Der and B. S. Everitt) Software for Data Analysis: Programming with R (J. M. Chambers) R for SAS and SPSS Users (R. A. Muenchen) Recent Advances in Linear Models and Related Areas: Essays in Honour of Helge Toutenburg (Shalabh and C. Heumann) Medical Statistics from A to Z, 2nd edition (B. Everitt) BAILEY, R. A. Design of Comparative Experiments (Cambridge Series in Statistical and Probabilistic Mathematics).

X 1 , X 2 ,. .. , X p be random variables called predictors (or inputs, covariates). Let X 1 , X 2 ,. .. , X p be their domains. We write shortly X := (X 1 , X 2 ,. .. , X p) for the vector of random predictor variables and X := X 1 × X 2 × • • • × X p for its domain. Y be a random variable called target (or output, response). Let Y be its domain. D ⊆ P(X × Y) be a (multi)set of instances of the unknown joint distribution p(X, Y) of predictors and target called data. D is often written as enumeration D = {(x 1 , y 1), (x 2 , y 2),. .. , (x n , y n)}

2012 IEEE International Workshop on Machine Learning for Signal Processing, 2012

This paper presents the application of the kernel signal to noise ratio (KSNR) in the context of feature extraction to general machine learning and signal processing domains. The proposed approach maximizes the signal variance while minimizes the estimated noise variance in a reproducing kernel Hilbert space (RKHS). The KSNR can be used in any kernel method to deal with correlated (possibly non-Gaussian) noise. We illustrate the method in nonlinear regression examples, dependence estimation and causal inference, nonlinear channel equalization, and nonlinear feature extraction from highdimensional satellite images. Results show that the proposed KSNR yields more fitted solutions and extracts more noisefree features when confronted with standard approaches.

2019

The Linear Regression (LR) model is arguably the most widely used statistical model in empirical modeling across many disciplines. It provides the exemplar for all regression models as well as several other statistical models referred to as ‘regression-like’ models, some of which will be discussed briefly in this chapter. The primary objective is to discuss the LR model and its associated statistical inference procedures. Special attention is paid to the model assumptions and how they relate to the sampling distributions of the statistics of interest. The main lesson of this chapter is that when any of the probabilistic assumptions of the LR model are invalid for data z0:={( )  =1  }  inferences based on it will be unreliable. The unreliability of inference will often stem from inconsistent estimators and sizeable discrepancies between actual and nominal error probabilities induced by statistical misspecification.

An important objective in scientific research and in more mundane data analysis tasks concerns the possibility of predicting the value of a dependent random variable based on the values of other independent variables, establishing a functional relation of a statistical nature. The study of such functional relations, known for historical reasons as regressions, goes back to pioneering works in Statistics.

Statistical learning refers to a set of tools for modeling and understanding complex datasets. It is a recently developed area in statistics, and blends with parallel developments in computer science, and in particular machine learning. The field encompasses many methods such as the lasso and sparse regression, classification and regression trees, and boosting and support vector machines.

References 395 Index 407 c 2020 M. P. Deisenroth, A. A. Faisal, C. S. Ong. To be published by Cambridge University Press.

To get a flavour of this book Project you will find a preliminary draft of chapter 5.