Papers by Jorge Magnun Santos
arXiv (Cornell University), Dec 6, 2017
Transfer Learning (TL) aims to transfer knowledge acquired in one problem, the source problem, on... more Transfer Learning (TL) aims to transfer knowledge acquired in one problem, the source problem, onto another problem, the target problem, dispensing with the bottom-up construction of the target model. Due to its relevance, TL has gained significant interest in the Machine Learning (ML) community since it paves the way to devise intelligent learning models that can easily be tailored to many different applications. As it is natural in a fast evolving area, a wide variety of TL methods, settings and nomenclature have been proposed so far. However, a wide range of works have been reporting different names for the same concepts. This concept and terminology mixture contribute however to obscure the TL field, hindering its proper consideration. In this paper we present a review of the literature on the majority of classification TL methods, and also a distribution-based categorization of TL with a common nomenclature suitable to classification problems. Under this perspective three main TL categories are presented, discussed and illustrated with examples.
Lecture Notes in Computer Science, 2005
The use of entropy as a cost function in the neural network learning phase usually implies that, ... more The use of entropy as a cost function in the neural network learning phase usually implies that, in the back-propagation algorithm, the training is done in batch mode. Apart from the higher complexity of the algorithm in batch mode, we know that this approach has some limitations over the sequential mode. In this paper we present a way of combining both modes when using entropic criteria. We present some experiments that validates the proposed method and we also show some comparisons of this proposed method with the single batch mode algorithm.
Biological and Artificial Intelligence Environments, 2005
One way of using the entropy criteria in learning systems is to minimize the entropy of the error... more One way of using the entropy criteria in learning systems is to minimize the entropy of the error between two variables: typically, one is the output of the learning system and the other is the target. This framework has been used for regression. In this paper we show how to use the minimization of the entropy of the error for classification. The minimization of the entropy of the error implies a constant value for the errors. This, in general, does not imply that the value of the errors is zero. In regression, this problem is solved by making a shift of the final result such that it's average equals the average value of the desired target. We prove that, under mild conditions, this algorithm, when used in a classification problem, makes the error converge to zero and can thus be used in classification.
The use of monolithic neural networks (such as a multilayer perceptron) has some drawbacks: e.g. ... more The use of monolithic neural networks (such as a multilayer perceptron) has some drawbacks: e.g. slow learning, weight coupling, the black box effect. These can be alleviated by the use of a modular neural network. The creation of a MNN has three steps: task decomposition, module creation and decision integration. In this paper we propose the use of an entropic clustering algorithm as a way of performing task decomposition. We present experiments on several real world classification problems that show the performance of this approach.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
Hierarchical clustering is a stepwise clustering method usually based on proximity measures betwe... more Hierarchical clustering is a stepwise clustering method usually based on proximity measures between objects or sets of objects from a given data set. The most common proximity measures are distance measures. The derived proximity matrices can be used to build graphs, which provide the basic structure for some clustering methods. We present here a new proximity matrix based on an entropic measure and also a clustering algorithm (LEGClust) that builds layers of subgraphs based on this matrix and uses them and a hierarchical agglomerative clustering technique to form the clusters. Our approach capitalizes on both a graph structure and a hierarchical construction. Moreover, by using entropy as a proximity measure, we are able, with no assumption about the cluster shapes, to capture the local structure of the data, forcing the clustering method to reflect this structure. We present several experiments on artificial and real data sets that provide evidence on the superior performance of this new algorithm when compared with competing ones.
Uploads
Papers by Jorge Magnun Santos