Skip to main content

Jiří Grim

Academy of Sciences of the Czech Republic, Pattern Recognition, Research Fellow

Followers

15

Following

8

Co-authors

6

Public Views

Robert Vandermeulen

Technical University of Denmark (DTU)

University of Economics Prague

University of Hertfordshire

Pierre Vandekerkhove

Nohemi Magallanes

Universidad Iberoamericana - Mexico

Duke University

University of Nottingham, China Campus

Interests

Uploads

Papers by Jiří Grim

Preprocessing of screening mammograms based on local statistical models

Proceedings of the 4th International Symposium on Applied Sciences in Biomedical and Communication Technologies - ISABEL '11, 2011

In a recent paper we have proposed evaluation of screening mammograms by means of local statistic... more In a recent paper we have proposed evaluation of screening mammograms by means of local statistical model. The model describes local statistical properties of internal pixels of a small search window -as they occur when scanning the mammogram. It is defined as a mixture of Gaussian densities which can be estimated by EM algorithm from data obtained by shifting the search window. The estimated Gaussian mixture is used to compute at all window positions the log-likelihood values which can be displayed as grey levels at the respective window centers. The resulting log-likelihood image closely correlates with the structural details of the original mammogram and emphasizes potential malignant findings as untypical locations of high novelty. In this paper we discuss the possibilities to enhance the log-likelihood image for diagnostic purposes.

Knowledge Representation and Uncertainty Processing in the Probabilistic Expert System Pes

International Journal of General Systems, 1993

The input and output information of the probabilistic expert system PES is expressed in terms of ... more The input and output information of the probabilistic expert system PES is expressed in terms of discrete random variables, the uncertainty of variables is characterized by discrete probability distributions. The knowledge base of PES has the form of a finite distribution mixture of product components which enables a simple computation of marginal and conditional distributions. The inference mechanism is fully defined by the formula of complete probability and reduces to conditional probability distribution in case of definite input information. The expert system PES was implemented on PC and applied to several problems.

An Iterative Inference Mechanism for the Probabilistic Expert System Pes

International Journal of General Systems, 1999

The paper suggests a new inference mechanism based on iterative use of the Bayesian inference sch... more The paper suggests a new inference mechanism based on iterative use of the Bayesian inference scheme. The procedure iteratively computes optimal component weights of a distribution mixture from a class called generalized knowledge base. It is proved that the iterative process converges to a unique limit whereby the resulting probability distribution can be defined as the information-divergence projection of input

Structural poisson mixtures for classification of documents

2008 19th International Conference on Pattern Recognition, 2008

Considering the statistical text classification problem we approximate class-conditional probabil... more Considering the statistical text classification problem we approximate class-conditional probability distributions by structurally modified Poisson mixtures. By introducing the structural model we can use different subsets of input variables to evaluate conditional probabilities of different classes in the Bayes formula. The method is applicable to document vectors of arbitrary dimension without any preprocessing. The structural optimization can be included into the EM algorithm in a statistically correct way.

A Discrete Mixtures Colour Texture Model

by Jiří Grim and Michal Haindl

A novel method of texture modelling based on discrete distribution mixtures is discussed. The sta... more A novel method of texture modelling based on discrete distribution mixtures is discussed. The statistical properties of textures are modelled by a discrete distribution mixture of product components. A natural colour texture is spectrally factorized and discrete mixtures models are learned and used to synthesize single orthogonal monospectral components. Texture synthesis is based on easy computation of arbitrary conditional distributions from the model. Finally single synthesized monospectral texture components are transformed into the required synthetic colour texture. The advantages and weak points of the presented approach are demonstrated on several colour texture applications.

A PROBABILISTIC APPROACH TO ROUGH TEXTURE COMPRESSION AND RENDERING

by Michal Haindl and Jiří Grim

Rough textures describe a general visual appearance of real-world materials with regard to view a... more Rough textures describe a general visual appearance of real-world materials with regard to view and illumination direc-tions. As the massive size and dimensionality of such repre-sentations is a main limitation of their broader use, efficient parameterization and compression methods are needed. Our method is based on estimating the joint probability density of the included nine spatial, directional, and spectral variables in the form of a Gaussian mixture of product components. Our reflectance prediction formula can be expressed analyti-cally as a simple continuous function of input variables and allows fast analytic evaluation for arbitrary spatial and direc-tional values without need for a lengthy interpolation from a finite grid of angular measurements. This method achieves high compression ratio increasing linearly with texture spatial resolution.

Probabilistic mixture-based image modelling

by Jiří Grim, Michal Haindl, and Vojtech Havlícek

Kybernetika -Praha-

During the last decade we have introduced probabilistic mixture models into image modelling area,... more During the last decade we have introduced probabilistic mixture models into image modelling area, which present highly atypical and extremely demanding applications for these models. This difficulty arises from the necessity to model tens thousands corre-lated data simultaneously and to reliably learn such unusually complex mixture models. Presented paper surveys these novel generative colour image models based on multivari-ate discrete, Gaussian or Bernoulli mixtures, respectively and demonstrates their major advantages and drawbacks on texture modelling applications. Our mixture models are restricted to represent two-dimensional visual information. Thus a measured 3D multi-spectral texture is spectrally factorized and corresponding multivariate mixture models are further learned from single orthogonal mono-spectral components and used to syn-thesise and enlarge these mono-spectral factor components. Texture synthesis is based on easy computation of arbitrary conditional distributi...

Computational Properties of Probabilistic Neural Networks

Lecture Notes in Computer Science, 2010

We discuss the problem of overfitting of probabilistic neural networks in the framework of statis... more We discuss the problem of overfitting of probabilistic neural networks in the framework of statistical pattern recognition. The probabilistic approach to neural networks provides a statistically justified subspace method of classification. The underlying structural mixture model includes binary structural parameters and can be optimized by EM algorithm in full generality. Formally, the structural model reduces the number of parameters included and therefore the structural mixtures become less complex and less prone to overfitting. We illustrate how recognition accuracy and the effect of overfitting is influenced by mixture complexity and by the size of training data set.

Recurrent Bayesian Reasoning in Probabilistic Neural Networks

Lecture Notes in Computer Science, 2007

Considering the probabilistic approach to neural networks in the framework of statistical pattern... more Considering the probabilistic approach to neural networks in the framework of statistical pattern recognition we assume approximation of class-conditional probability distributions by finite mixtures of product components. The mixture components can be interpreted as probabilistic neurons in neurophysiological terms and, in this respect, the fixed probabilistic description becomes conflicting with the well known short-term dynamic properties of biological neurons. We show that some parameters of PNN can be "released" for the sake of dynamic processes without destroying the statistically correct decision making. In particular, we can iteratively adapt the mixture component weights or modify the input pattern in order to facilitate the correct recognition.

Recognition of Properties by Probabilistic Neural Networks

Lecture Notes in Computer Science, 2009

The statistical pattern recognition based on Bayes formula implies the concept of mutually exclus... more The statistical pattern recognition based on Bayes formula implies the concept of mutually exclusive classes. This assumption is not applicable when we have to identify some non-exclusive properties and therefore it is unnatural in biological neural networks. Considering the framework of probabilistic neural networks we propose statistical identification of non-exclusive properties by using one-class classifiers.

Minimum Information Loss Cluster Analysis for Categorical Data

Lecture Notes in Computer Science, 2007

The EM algorithm has been used repeatedly to identify latent classes in categorical data by estim... more The EM algorithm has been used repeatedly to identify latent classes in categorical data by estimating finite distribution mixtures of product components. Unfortunately, the underlying mixtures are not uniquely identifiable and, moreover, the estimated mixture parameters are starting-point dependent. For this reason we use the latent class model only to define a set of "elementary" classes by estimating a mixture of a large number components. We propose a hierarchical "bottom up" cluster analysis based on unifying the elementary latent classes sequentially. The clustering procedure is controlled by minimum information loss criterion.

Branch & Bound Algorithm with Partial Prediction for Use with Recursive and Non-Recursive Criterion Forms

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 2001

We introduce a novel algorithm for optimal feature selection. As opposed to our recent Fast Branc... more We introduce a novel algorithm for optimal feature selection. As opposed to our recent Fast Branch &amp;amp;amp; Bound (FBB) algorithm [5] the new algorithm is well suitable for use with recursive criterion forms. Even if the new algorithm does not operate as effectively as the FBB algorithm, it is able to find the optimum significantly faster than any other Branch &amp;amp;amp; Bound [1],[3] algorithm.

The Problem of Fragile Feature Subset Preference in Feature Selection Methods and a Proposal of Algorithmic Workaround

by Pavel Pudil and Jiří Grim

2010 20th International Conference on Pattern Recognition, 2010

We point out a problem inherent in the optimization scheme of many popular feature selection meth... more We point out a problem inherent in the optimization scheme of many popular feature selection methods. It follows from the implicit assumption that higher feature selection criterion value always indicates more preferable subset even if the value difference is marginal. This assumption ignores the reliability issues of particular feature preferences, overfitting and feature acquisition cost. We propose an algorithmic extension applicable to many standard feature selection methods allowing better control over feature subset preference. We show experimentally that the proposed mechanism is capable of reducing the size of selected subsets as well as improving classifier generalization.

A Gaussian mixture-based colour texture model

by Jiří Grim, Michal Haindl, and Pavel Pudil

Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., 2004

A new method of colour texture modelling based on Gaussian distribution mixtures is discussed. We... more A new method of colour texture modelling based on Gaussian distribution mixtures is discussed. We estimate the local statistical properties of the monospectral version of the target texture in the form of a Gaussian mixture of product components. The synthesized texture is obtained by means of a step-wise prediction of the texture image. In order to achieve a realistic colour texture image and to avoid possible loss of high-frequency details we use optimally chosen pieces of the original colour source texture in the synthesis phase. In this sense the proposed texture modelling method can be viewed as a statistically controlled sampling. By using multispectral or mutually registered BTF texture pieces the method can be easily extended also for these textures.

On virtually binary nature of probabilistic neural networks

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 1998

Parameters and interconnection structure of multilayered probabilistic neural networks can be opt... more Parameters and interconnection structure of multilayered probabilistic neural networks can be optimized layer-by-layer by estimating unknown probability distributions on input space in the form of finite distribution mixtures. The components of mixtures correspond to neurons which perform an information preserving transform between consecutive layers. Simultaneously the entropy of the transformed distribution is minimized. It is argued that in multidimensional spaces and particularly at higher levels of multilayer feedforward neural networks, the output variables of probabilistic neurons tend to be binary. It is shown that the information loss caused by the binary approximation of neurons can be suppressed by increasing the approximation accuracy.

Information Analysis of Multiple Classifier Fusion?

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 2001

We consider a general scheme of parallel classifier combinations in the framework of statistical ... more We consider a general scheme of parallel classifier combinations in the framework of statistical pattern recognition. Each statistical classifier defines a set of output variables in terms of a posteriori probabilities, i.e. it is used as a feature extractor. Unlike usual combining schemes the output vectors of classifiers are combined in parallel. The statistical Shannon information is used as a criterion to compare different combining schemes from the point of view of the theoretically available decision information. By means of relatively simple arguments we derive a theoretical hierarchy between different schemes of classifier fusion in terms of information inequalities.

Boosting in probabilistic neural networks

by Pavel Pudil and Jiří Grim

Object recognition supported by user interaction for service robots, 2002

The basic idea of boosting is to increase the pattern recognition accuracy by combining classifie... more The basic idea of boosting is to increase the pattern recognition accuracy by combining classifiers which have been derived from differently weighted versions of the original training data. It has been verified in practical experiments that the resulting classification performance can be improved by increasing the weights of misclassified training samples. However, in statistical pattern recognition, the weighted data may influence the form of the estimated conditional distributions and therefore the theoretically achievable classification error could increase. We prove that in case of maximum-likelihood estimation the weighting of discrete data vectors is asymptotically equivalent to multiplication of the estimated discrete conditional distributions by a positive bounded function. Consequently, the Bayesian decision-making is shown to be asymptotically invariant with respect to arbitrary weighting of data provided that (a) the weighting function is defined identically for all classes and (b) the prior probabilities are properly modified.

Criteria Ensembles in Feature Selection

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 2009

ABSTRACT In feature selection the effect of over-fitting may lead to serious degradation of gener... more ABSTRACT In feature selection the effect of over-fitting may lead to serious degradation of generalization ability. We introduce the concept of combining multiple feature selection criteria in feature selection methods with the aim to obtain feature subsets that generalize better. The concept is applicable with many existing feature selection methods. Here we discuss in more detail the family of sequential search methods. The concept does not specify which criteria to combine – to illustrate its feasibility we give a simple example of combining the estimated accuracy of k-nearest neighbor classifiers for various k. We perform the experiments on a number of datasets. The potential to improve is clearly seen on improved classifier performance on independent test data as well as on improved feature selection stability.

Multivariate structural Bernoulli mixtures for recognition of handwritten numerals

by Pavel Pudil and Jiří Grim

Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, 2000

As shown recently, the structural optimization of probabilistic neural networks can be included i... more As shown recently, the structural optimization of probabilistic neural networks can be included into EM algorithm by introducing a special type of multivariate Bernoulli mixtures. However, the underlying loglikelihood criterion is known to be multimodal in case of mixtures and therefore the EM iteration process may be starting-point dependent. In the present paper we discuss the possibility of a proper initialization of EM algorithm by means of a nonparametric optimally smoothed kernel estimate of the unknown probability distribution. The method has been applied to recognize unconstrained handwritten numerals from the database of Concordia University in Montreal.

On Prediction Mechanisms in Fast Branch & Bound Algorithms

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 2004

The idea of using the Branch & Bound search for optimal feature selection has been recently refin... more The idea of using the Branch & Bound search for optimal feature selection has been recently refined by introducing additional predicting heuristics that is able to considerably accelerate the search process while keeping the optimality of results unaffected. The heuristics is used most extensively in the so-called Fast Branch & Bound algorithm, where it replaces many slow criterion function computations by means of fast predictions. In this paper we investigate alternative prediction mechanisms. The alternatives are shown potentially useful for simplification and speed-up of the algorithm. We demonstrate the robustness of the prediction mechanism concept on real data experiments.

Preprocessing of screening mammograms based on local statistical models

Proceedings of the 4th International Symposium on Applied Sciences in Biomedical and Communication Technologies - ISABEL '11, 2011

In a recent paper we have proposed evaluation of screening mammograms by means of local statistic... more In a recent paper we have proposed evaluation of screening mammograms by means of local statistical model. The model describes local statistical properties of internal pixels of a small search window -as they occur when scanning the mammogram. It is defined as a mixture of Gaussian densities which can be estimated by EM algorithm from data obtained by shifting the search window. The estimated Gaussian mixture is used to compute at all window positions the log-likelihood values which can be displayed as grey levels at the respective window centers. The resulting log-likelihood image closely correlates with the structural details of the original mammogram and emphasizes potential malignant findings as untypical locations of high novelty. In this paper we discuss the possibilities to enhance the log-likelihood image for diagnostic purposes.

Knowledge Representation and Uncertainty Processing in the Probabilistic Expert System Pes

International Journal of General Systems, 1993

The input and output information of the probabilistic expert system PES is expressed in terms of ... more The input and output information of the probabilistic expert system PES is expressed in terms of discrete random variables, the uncertainty of variables is characterized by discrete probability distributions. The knowledge base of PES has the form of a finite distribution mixture of product components which enables a simple computation of marginal and conditional distributions. The inference mechanism is fully defined by the formula of complete probability and reduces to conditional probability distribution in case of definite input information. The expert system PES was implemented on PC and applied to several problems.

An Iterative Inference Mechanism for the Probabilistic Expert System Pes

International Journal of General Systems, 1999

The paper suggests a new inference mechanism based on iterative use of the Bayesian inference sch... more The paper suggests a new inference mechanism based on iterative use of the Bayesian inference scheme. The procedure iteratively computes optimal component weights of a distribution mixture from a class called generalized knowledge base. It is proved that the iterative process converges to a unique limit whereby the resulting probability distribution can be defined as the information-divergence projection of input

Structural poisson mixtures for classification of documents

2008 19th International Conference on Pattern Recognition, 2008

Considering the statistical text classification problem we approximate class-conditional probabil... more Considering the statistical text classification problem we approximate class-conditional probability distributions by structurally modified Poisson mixtures. By introducing the structural model we can use different subsets of input variables to evaluate conditional probabilities of different classes in the Bayes formula. The method is applicable to document vectors of arbitrary dimension without any preprocessing. The structural optimization can be included into the EM algorithm in a statistically correct way.

A Discrete Mixtures Colour Texture Model

by Jiří Grim and Michal Haindl

A novel method of texture modelling based on discrete distribution mixtures is discussed. The sta... more A novel method of texture modelling based on discrete distribution mixtures is discussed. The statistical properties of textures are modelled by a discrete distribution mixture of product components. A natural colour texture is spectrally factorized and discrete mixtures models are learned and used to synthesize single orthogonal monospectral components. Texture synthesis is based on easy computation of arbitrary conditional distributions from the model. Finally single synthesized monospectral texture components are transformed into the required synthetic colour texture. The advantages and weak points of the presented approach are demonstrated on several colour texture applications.

A PROBABILISTIC APPROACH TO ROUGH TEXTURE COMPRESSION AND RENDERING

by Michal Haindl and Jiří Grim

Rough textures describe a general visual appearance of real-world materials with regard to view a... more Rough textures describe a general visual appearance of real-world materials with regard to view and illumination direc-tions. As the massive size and dimensionality of such repre-sentations is a main limitation of their broader use, efficient parameterization and compression methods are needed. Our method is based on estimating the joint probability density of the included nine spatial, directional, and spectral variables in the form of a Gaussian mixture of product components. Our reflectance prediction formula can be expressed analyti-cally as a simple continuous function of input variables and allows fast analytic evaluation for arbitrary spatial and direc-tional values without need for a lengthy interpolation from a finite grid of angular measurements. This method achieves high compression ratio increasing linearly with texture spatial resolution.

Probabilistic mixture-based image modelling

by Jiří Grim, Michal Haindl, and Vojtech Havlícek

Kybernetika -Praha-

During the last decade we have introduced probabilistic mixture models into image modelling area,... more During the last decade we have introduced probabilistic mixture models into image modelling area, which present highly atypical and extremely demanding applications for these models. This difficulty arises from the necessity to model tens thousands corre-lated data simultaneously and to reliably learn such unusually complex mixture models. Presented paper surveys these novel generative colour image models based on multivari-ate discrete, Gaussian or Bernoulli mixtures, respectively and demonstrates their major advantages and drawbacks on texture modelling applications. Our mixture models are restricted to represent two-dimensional visual information. Thus a measured 3D multi-spectral texture is spectrally factorized and corresponding multivariate mixture models are further learned from single orthogonal mono-spectral components and used to syn-thesise and enlarge these mono-spectral factor components. Texture synthesis is based on easy computation of arbitrary conditional distributi...

Computational Properties of Probabilistic Neural Networks

Lecture Notes in Computer Science, 2010

We discuss the problem of overfitting of probabilistic neural networks in the framework of statis... more We discuss the problem of overfitting of probabilistic neural networks in the framework of statistical pattern recognition. The probabilistic approach to neural networks provides a statistically justified subspace method of classification. The underlying structural mixture model includes binary structural parameters and can be optimized by EM algorithm in full generality. Formally, the structural model reduces the number of parameters included and therefore the structural mixtures become less complex and less prone to overfitting. We illustrate how recognition accuracy and the effect of overfitting is influenced by mixture complexity and by the size of training data set.

Recurrent Bayesian Reasoning in Probabilistic Neural Networks

Lecture Notes in Computer Science, 2007

Considering the probabilistic approach to neural networks in the framework of statistical pattern... more Considering the probabilistic approach to neural networks in the framework of statistical pattern recognition we assume approximation of class-conditional probability distributions by finite mixtures of product components. The mixture components can be interpreted as probabilistic neurons in neurophysiological terms and, in this respect, the fixed probabilistic description becomes conflicting with the well known short-term dynamic properties of biological neurons. We show that some parameters of PNN can be "released" for the sake of dynamic processes without destroying the statistically correct decision making. In particular, we can iteratively adapt the mixture component weights or modify the input pattern in order to facilitate the correct recognition.

Recognition of Properties by Probabilistic Neural Networks

Lecture Notes in Computer Science, 2009

The statistical pattern recognition based on Bayes formula implies the concept of mutually exclus... more The statistical pattern recognition based on Bayes formula implies the concept of mutually exclusive classes. This assumption is not applicable when we have to identify some non-exclusive properties and therefore it is unnatural in biological neural networks. Considering the framework of probabilistic neural networks we propose statistical identification of non-exclusive properties by using one-class classifiers.

Minimum Information Loss Cluster Analysis for Categorical Data

Lecture Notes in Computer Science, 2007

The EM algorithm has been used repeatedly to identify latent classes in categorical data by estim... more The EM algorithm has been used repeatedly to identify latent classes in categorical data by estimating finite distribution mixtures of product components. Unfortunately, the underlying mixtures are not uniquely identifiable and, moreover, the estimated mixture parameters are starting-point dependent. For this reason we use the latent class model only to define a set of "elementary" classes by estimating a mixture of a large number components. We propose a hierarchical "bottom up" cluster analysis based on unifying the elementary latent classes sequentially. The clustering procedure is controlled by minimum information loss criterion.

Branch & Bound Algorithm with Partial Prediction for Use with Recursive and Non-Recursive Criterion Forms

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 2001

We introduce a novel algorithm for optimal feature selection. As opposed to our recent Fast Branc... more We introduce a novel algorithm for optimal feature selection. As opposed to our recent Fast Branch &amp;amp;amp; Bound (FBB) algorithm [5] the new algorithm is well suitable for use with recursive criterion forms. Even if the new algorithm does not operate as effectively as the FBB algorithm, it is able to find the optimum significantly faster than any other Branch &amp;amp;amp; Bound [1],[3] algorithm.

The Problem of Fragile Feature Subset Preference in Feature Selection Methods and a Proposal of Algorithmic Workaround

by Pavel Pudil and Jiří Grim

2010 20th International Conference on Pattern Recognition, 2010

We point out a problem inherent in the optimization scheme of many popular feature selection meth... more We point out a problem inherent in the optimization scheme of many popular feature selection methods. It follows from the implicit assumption that higher feature selection criterion value always indicates more preferable subset even if the value difference is marginal. This assumption ignores the reliability issues of particular feature preferences, overfitting and feature acquisition cost. We propose an algorithmic extension applicable to many standard feature selection methods allowing better control over feature subset preference. We show experimentally that the proposed mechanism is capable of reducing the size of selected subsets as well as improving classifier generalization.

A Gaussian mixture-based colour texture model

by Jiří Grim, Michal Haindl, and Pavel Pudil

Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., 2004

A new method of colour texture modelling based on Gaussian distribution mixtures is discussed. We... more A new method of colour texture modelling based on Gaussian distribution mixtures is discussed. We estimate the local statistical properties of the monospectral version of the target texture in the form of a Gaussian mixture of product components. The synthesized texture is obtained by means of a step-wise prediction of the texture image. In order to achieve a realistic colour texture image and to avoid possible loss of high-frequency details we use optimally chosen pieces of the original colour source texture in the synthesis phase. In this sense the proposed texture modelling method can be viewed as a statistically controlled sampling. By using multispectral or mutually registered BTF texture pieces the method can be easily extended also for these textures.

On virtually binary nature of probabilistic neural networks

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 1998

Parameters and interconnection structure of multilayered probabilistic neural networks can be opt... more Parameters and interconnection structure of multilayered probabilistic neural networks can be optimized layer-by-layer by estimating unknown probability distributions on input space in the form of finite distribution mixtures. The components of mixtures correspond to neurons which perform an information preserving transform between consecutive layers. Simultaneously the entropy of the transformed distribution is minimized. It is argued that in multidimensional spaces and particularly at higher levels of multilayer feedforward neural networks, the output variables of probabilistic neurons tend to be binary. It is shown that the information loss caused by the binary approximation of neurons can be suppressed by increasing the approximation accuracy.

Information Analysis of Multiple Classifier Fusion?

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 2001

We consider a general scheme of parallel classifier combinations in the framework of statistical ... more We consider a general scheme of parallel classifier combinations in the framework of statistical pattern recognition. Each statistical classifier defines a set of output variables in terms of a posteriori probabilities, i.e. it is used as a feature extractor. Unlike usual combining schemes the output vectors of classifiers are combined in parallel. The statistical Shannon information is used as a criterion to compare different combining schemes from the point of view of the theoretically available decision information. By means of relatively simple arguments we derive a theoretical hierarchy between different schemes of classifier fusion in terms of information inequalities.

Boosting in probabilistic neural networks

by Pavel Pudil and Jiří Grim

Object recognition supported by user interaction for service robots, 2002

The basic idea of boosting is to increase the pattern recognition accuracy by combining classifie... more The basic idea of boosting is to increase the pattern recognition accuracy by combining classifiers which have been derived from differently weighted versions of the original training data. It has been verified in practical experiments that the resulting classification performance can be improved by increasing the weights of misclassified training samples. However, in statistical pattern recognition, the weighted data may influence the form of the estimated conditional distributions and therefore the theoretically achievable classification error could increase. We prove that in case of maximum-likelihood estimation the weighting of discrete data vectors is asymptotically equivalent to multiplication of the estimated discrete conditional distributions by a positive bounded function. Consequently, the Bayesian decision-making is shown to be asymptotically invariant with respect to arbitrary weighting of data provided that (a) the weighting function is defined identically for all classes and (b) the prior probabilities are properly modified.

Criteria Ensembles in Feature Selection

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 2009

ABSTRACT In feature selection the effect of over-fitting may lead to serious degradation of gener... more ABSTRACT In feature selection the effect of over-fitting may lead to serious degradation of generalization ability. We introduce the concept of combining multiple feature selection criteria in feature selection methods with the aim to obtain feature subsets that generalize better. The concept is applicable with many existing feature selection methods. Here we discuss in more detail the family of sequential search methods. The concept does not specify which criteria to combine – to illustrate its feasibility we give a simple example of combining the estimated accuracy of k-nearest neighbor classifiers for various k. We perform the experiments on a number of datasets. The potential to improve is clearly seen on improved classifier performance on independent test data as well as on improved feature selection stability.

Multivariate structural Bernoulli mixtures for recognition of handwritten numerals

by Pavel Pudil and Jiří Grim

Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, 2000

As shown recently, the structural optimization of probabilistic neural networks can be included i... more As shown recently, the structural optimization of probabilistic neural networks can be included into EM algorithm by introducing a special type of multivariate Bernoulli mixtures. However, the underlying loglikelihood criterion is known to be multimodal in case of mixtures and therefore the EM iteration process may be starting-point dependent. In the present paper we discuss the possibility of a proper initialization of EM algorithm by means of a nonparametric optimally smoothed kernel estimate of the unknown probability distribution. The method has been applied to recognize unconstrained handwritten numerals from the database of Concordia University in Montreal.

On Prediction Mechanisms in Fast Branch & Bound Algorithms

by Pavel Pudil and Jiří Grim

Lecture Notes in Computer Science, 2004

The idea of using the Branch & Bound search for optimal feature selection has been recently refin... more The idea of using the Branch & Bound search for optimal feature selection has been recently refined by introducing additional predicting heuristics that is able to considerably accelerate the search process while keeping the optimality of results unaffected. The heuristics is used most extensively in the so-called Fast Branch & Bound algorithm, where it replaces many slow criterion function computations by means of fast predictions. In this paper we investigate alternative prediction mechanisms. The alternatives are shown potentially useful for simplification and speed-up of the algorithm. We demonstrate the robustness of the prediction mechanism concept on real data experiments.

EM Cluster Analysis for Categorical Data

Product Mixture: Conditional Independence Model discrete random variables: ξ n ∈ X n , n ∈ N , N ... more Product Mixture: Conditional Independence Model discrete random variables: ξ n ∈ X n , n ∈ N , N = {1, . . . , N} X n : finite sets of categorical values (no ordering) random vector: ξ = (ξ 1 , . . . , ξ N ) ∈ X , X = X 1 × · · · × X N discrete random variable: µ ∈ M, M = {1, . . . , M} P{µ = m} = w m , m ∈ M, m∈M

Texture Oriented Image Inpainting Based on Local Statistical Model

by Jiří Grim and M. Malec

Image inpainting as a means of substituting missing image parts can become difficult when the %mi... more Image inpainting as a means of substituting missing image parts can
become difficult when the %missing parts of the image are textured.
image is textured. In this paper we apply a local statistical model of the source color image with the aim to predict missing texture regions. We have shown in a series of papers that textures can be modeled locally by
estimating the joint probability density of spectral pixel values in a suitably chosen observation window. For the sake of image inpainting we estimate the joint multivariate density in the form of a Gaussian mixture of product components. The missing image region is inpainted iteratively by step-wise prediction of the unknown spectral values.

Digital Image Forgery Detection by Local Statistical Models

Distribution Mixtures of Product Components Part I: EM Algorithm & Modifications