Functional Links

Due to the prevalence of unlabeled data, semisupervised learning has drawn significant attention and has been found applicable in many realworld applications. In this paper, we present the so-called Budgeted Semi-supervised Support Vector Machine (BS3VM), a method that leverages the excellent generalization capacity of kernel-based method with the adjacent and distributive information carried in a spectral graph for semi-supervised learning purpose. The fact that the optimization problem of BS3VM can be solved directly in the primal form makes it fast and efficient in memory usage. We validate the proposed method on several benchmark datasets to demonstrate its accuracy and efficiency. The experimental results show that BS3VM can scale up efficiently to the large-scale datasets where it yields a comparable classification accuracy while simultaneously achieving a significant computational speed-up compared with the baselines.

The main aim of the article is to clarify the characteristics of the method that determines the functional links and gravity towards cities on the basis of public transport connections. The paper delivers a detailed description of the method and provides examples of its application in empirical research conducted by the authors in the years 2010-2014.

h i g h l i g h t s • This paper proposes an improved split functional link adaptive filter (SFLAF). • The proposed model is characterized by the adaptive combination of two APA filters. • An advanced scheme is also proposed involving the combination of multiple filters. • The adaptive combinations are performed for all the projections of the APA filters. • The proposed models are assessed in three different nonlinear modeling problems.

Background: Optimal treatment for cancer patients relies on accurate pathological diagnosis. The world health organization has defined almost 100 known central nervous system (CNS) tumors, making the histopathological diagnostic process particularly challenging. To help reduce variability, validation costs, and standardize the histopathological diagnostic process, supervised machine learning techniques using epigenetic data have been developed. These methods require large labeled training data sets to obtain clinically acceptable classification accuracy. However, labeling pathology data for machine learning models is time-consuming and resource-intensive, especially for rare tumor types. On the other hand, there is now abundance of unlabeled epigenetic data across multiple databases. To maximize the utility of both labeled and unlabeled data, semi-supervised learning (SSL) approach has been proposed and shown to be effective in genomic field. However, it has not yet been explored wi...

Nonlinear models are known to provide excellent performance in real-world applications that often operate in non-ideal conditions. However, such applications often require online processing to be performed with limited computational resources. In this paper, we propose a new efficient nonlinear model for online applications. The proposed algorithm is based on the linear-in-the-parameters (LIP) nonlinear filters and their implementation as functional link adaptive filters (FLAFs). We focus here on a new effective and efficient approach for FLAFs based on frequency-domain adaptive filters. We introduce the class of frequency-domain functional link adaptive filters (FD-FLAFs) and propose a partitioned block approach for their implementation. We also investigate on the functional link expansions that provide the most significant benefits operating with limited resources in the frequency-domain. We present and compare FD-FLAFs with different expansions to identify the LIP nonlinear filters showing the best tradeoff between performance and computational complexity. Experimental results prove that the frequency domain LIP nonlinear filters can be considered as an efficient and effective solution for online applications, like the nonlinear acoustic echo cancellation.

The paper proposes a novel approach for measuring the room impulse response that is robust toward the nonlinearities affecting the power amplifier or the loudspeaker. The approach is implemented by modeling the acoustic path as a Legendre nonlinear filter and by measuring the first-order kernel using perfect periodic sequences and the cross-correlation method. Perfect sequences for Legendre filters are periodic sequences that guarantee the orthogonality of the Legendre basis functions over a period. They ensure the robustness of the first kernel measurement toward nonlinear distortions. The paper also explains how perfect periodic sequences for Legendre filters that are suitable for room impulse response identification can be developed. Experimental results involving both simulated and real environments illustrate the effectiveness and the robustness of the proposed approach.

We investigated adaptive algorithms for a Hammerstein block structure in which a static nonlinear block and dynamic linear block are cascaded. The approach considered here is to use generalized orthonormal basis functions in a Hammerstein block structure by using fixed pole filter banks. We applied the normalized least mean square approach to the developed adaptive algorithm in order to acquire Hammerstein block structure parameters. Performance comparison of the proposed approach was investigated considering convergence speed and parametric complexity for acoustic echo cancellation application. The results indicated that in the developed algorithm along with appropriate selection of fixed poles, the algorithm convergences faster and less parametric complexity is provided when compared to direct adaptive Hammerstein algorithms with IIR and FIR linear blocks.

Nonlinearities in the amplifier and loudspeaker of hands-free speakerphones limit the performance of linear adaptive acoustic echo cancellers, necessitating the use of nonlinear cancellation schemes. A nonlinear acoustic echo canceller based on the WienerHammerstein model structure of a cascade of linear, memoryless nonlinear, and linear elements is proposed. By modeling the true structure of the nonlinear system, the proposed canceller requires relatively few adaptive parameters, offering significantly lower storage and computational requirements than more general nonlinear adaptive filtering techniques. Experimental results on measured loudspeaker signals indicate that the proposed nonlinear echo canceller provides as much as an 8.4 dB improvement in Echo Return Loss Enhancement (ERLE) over a linear Normalized LMS canceller with little additional computation.

In this paper, an Incremental Neural Network for Classification and Clustering (INNCC) is proposed. The main advantages of this neural network are the linkage between data topology preservation and classes representation by using the cluster posterior probabilities of classes. It is a constructive model without prior conditions such as a suitable number of nodes. A new neuron is inserted when new data are not represented by existing neurons. In training step, both supervised and unsupervised learning are used. The training dataset contains few samples with class labels and several unlabeled ones. The Support Vector Machines (SVM) operates in the training step to assess the INNCC classification result. The proposed approach has been tested on synthetic and real datasets. Obtained results are very promising.

The performance of adaptive acoustic echo cancelers (AEC) is sensitive to the non-stationarity and correlation of speech signals. In this article, we explore a new approach based on an adaptive AEC driven by data hidden in speech, to enhance the AEC robustness. We propose a two-stage AEC, where the first stage is a classical NLMS-based AEC driven by the farend speech. In the signal, we embed-in an extended conception of data hiding-an imperceptible white and stationary signal, i.e. a watermark. The goal of the second stage AEC is to identify the misalignment of the first stage. It is driven by the watermark solely, and takes advantage of its appropriate properties (stationary and white) to improve the robustness of the two-stage AEC to the non-stationarity and correlation of speech, and thus reduce the overall system misadjustment. We test two kinds of implementations: in the first implementation, referred to as A-WdAEC (Adaptive Watermark driven AEC), the watermark is a white stationary Gaussian noise. Driven by this signal, the second stage converges faster than the classical AEC and provides better performance in steady state. In the second implementation, referred to as MLS-WdAEC, the watermark is built from maximum length sequences (MLS). Thus, the second stage performs a block identification of the first stage misalignment, given by the circular correlation watermark/preprocessed version of the first stage residual echo. The advantage of this implementation lies in its robustness against noise and under-modeling. Simulation results show the relevance of the "watermark-driven AEC" approach, compared to the classical "error driven AEC".

The main aim of the article is to clarify the characteristics of the method that determines the functional links and gravity towards cities on the basis of public transport connections. The paper delivers a detailed description of the method and provides examples of its application in empirical research conducted by the authors in the years 2010-2014.

With the rapid development of social media sharing, people often need to manage the growing volume of multimedia data such as large scale video classification and annotation, especially to organize those videos containing human activities. Recently, manifold regularized semi-supervised learning (SSL), which explores the intrinsic data probability distribution and then improves the generalization ability with only a small number of labeled data, has emerged as a promising paradigm for semiautomatic video classification. In addition, human action videos often have multi-modal content and different representations. To tackle the above problems, in this paper we propose multiview Hessian regularized logistic regression (mHLR) for human action recognition. Compared with existing work, the advantages of mHLR lie in three folds: (1) mHLR combines multiple Hessian regularization, each of which obtained from a particular representation of instance, to leverage the exploring of local geometry; (2) mHLR naturally handle multi-view instances with multiple representations; (3) mHLR employs a smooth loss function and then can be effectively optimized. We carefully conduct extensive experiments on the unstructured social activity attribute (USAA) dataset and the experimental results demonstrate the effectiveness of the proposed multiview Hessian regularized logistic regression for human action recognition. Highlights: (1) mHLR combines multiple Hessian regularization to leverage the exploring of local geometry; (2) mHLR can handle multi-view instances with multiple representations; (3) mHLR employs a smooth loss function and then can be effectively optimized.

The main aim of the article is to clarify the characteristics of the method that determines the functional links and gravity towards cities on the basis of public transport connections. The paper delivers a detailed description of the method and provides examples of its application in empirical research conducted by the authors in the years 2010-2014.

The main aim of the article is to clarify the characteristics of the method that determines the functional links and gravity towards cities on the basis of public transport connections. The paper delivers a detailed description of the method and provides examples of its application in empirical research conducted by the authors in the years 2010-2014.

Functional Link Artificial Neural Networks (FLANNs) have been extensively used for tasks of audio and speech classification, due to their combination of universal approximation capabilities and fast training. The performance of a FLANN, however, is known to be dependent on the specific functional link (FL) expansion that is used. In this paper, we provide an extensive benchmark of multiple FL expansions on several audio classification problems, including speech discrimination , genre classification, and artist recognition. Our experimental results show that a random-vector expansion is well suited for classification tasks, achieving the best accuracy in two out of three tasks.

Voting-based extreme learning machine (V-ELM) was proposed to improve learning efficiency where majority voting was employed. V-ELM assumes that all individual classifiers contribute equally to the decision ensemble. However, in many real-world scenarios, this assumption does not work well. In this paper, we aim to enhance V-ELM by introducing weights to distinguish the importance of each individual ELM classifier in decision making. Genetic algorithm is used for optimizing these weights. This evolutionary V-ELM is named as EV-ELM. Results on several benchmark databases show that EV-ELM achieves the highest classification accuracy compared with V-ELM and ELM.

In this paper, we investigate the problem of music classification when training data is distributed throughout a network of interconnected agents (e.g. computers, or mobile devices), and it is available in a sequential stream. Under the considered setting, the task is for all the nodes, after receiving any new chunk of training data, to agree on a single classifier in a decentralized fashion, without reliance on a master node. In particular, in this paper we propose a fully decentralized, sequential learning algorithm for a class of neural networks known as Random Vector Functional-Link nets. The proposed algorithm does not require the presence of a single  coordinating agent, and it is formulated exclusively in term of local exchanges between neighboring nodes, thus making it useful in a wide range of realistic situations. Experimental simulations on four music classification benchmarks show that the algorithm has comparable performance with respect to a centralized solution, where a single agent collects all the local data from every node and subsequently updates the model.

Nonlinear distortions pose a serious problem for the quality preservation of audio and speech signals. To address this problem, such signals are processed by nonlinear models.  Functional link adaptive filter (FLAF) is a Iinear-in-the-parameter
nonlinear model, whose nonlinear transformation of the input is characterized by a basis function expansion, satisfying the universal approximation properties. Since the expansion type affects the nonlinear modeling according to the nature of the input signal, in this paper we investigate the FLAF modeling performance involving the most popular functional expansions when audio and speech signals are processed. A  comprehensive analysis is conducted to provide the best suitable solution for the processing of nonlinear signals. Experimental results are assessed also in terms of signal quality and intelligibility.

Semi-supervised learning (SSL) is the problem of learning a function with only a partially labeled training set. It has considerable practical interest in applications where labeled data is costly to obtain, while unlabeled data is abundant. One approach to SSL in the case of binary classification is inspired by work on transductive learning (TL) by V. Vapnik. It has been applied prevalently using support vector machines (SVM) as the base learning algorithm, giving rise to the so-called transductive SVM (TR-SVM). The resulting optimization problem, however, is highly non-convex and complex to solve. In this paper, we propose an alternative semi-supervised training algorithm based on the TL theory, namely semi-supervised random vector functional-link (RVFL) network, which is able to obtain state-of-the-art performance, while resulting in a standard convex optimization problem. In particular we show that, thanks to the characteristics of RVFLs networks, the resulting optimization problem can be safely approximated with a standard quadratic programming problem solvable in polynomial time. A wide range of experiments validate our proposal. As a comparison, we also propose a semi-supervised algorithm for RVFLs based on the theory of manifold regularization.

One of the main characteristics in many real-world big data scenarios is their distributed nature. In a machine learning context, distributed data, together with the requirements of preserving privacy and scaling up to large networks, brings the challenge of designing fully decentralized training protocols. In this paper, we explore the problem of distributed learning when the features of every pattern are available throughout multiple agents (as is happening, for example, in a distributed database scenario). We propose an algorithm for a particular class of neural networks, known as Random Vector Functional-Link (RVFL), which is based on the Alternating Direction Method of Multipliers optimization algorithm. The proposed algorithm allows to learn an RVFL network from multiple distributed data sources, while restricting communication to the unique operation of computing a distributed average. Our experimental simulations show that the algorithm is able to achieve a generalization accuracy comparable to a fully centralized solution, while at the same time being extremely efficient.

This paper introduces a new method for improving nonlinear modeling performance in online learning by using functional link-based models. The proposed algorithm is capable of selecting the useful nonlinear elements resulting from the functional expansion, while setting to zero the ones that does not bring any improvement of the modeling performance. This allows to reduce any gradient noise due to a possible overestimate of the solution, thus preventing any overfitting phenomena. The proposed model is assessed in several nonlinear identification problems, including different levels of nonlinearity, showing significant improvements.

This paper aims to develop distributed learning algorithms for Random Vector Functional-Link (RVFL) networks, where training data is distributed under a decentralized information structure. Two algorithms are proposed by using Decentralized Average Consensus (DAC) and Alternating Direction Method of Multipliers (ADMM) strategies, respectively. These algorithms work in a fully distributed fashion and have no requirement on coordination from a central agent during the learning process. For distributed learning, the goal is to build a common learner model which optimizes the system performance over the whole set of local data. In this work, it is assumed that all stations know the initial weights of the input layer, the output weights of local RVFL networks can be shared through communication channels among neighboring nodes only, and local datasets are blocked strictly. The proposed learning algorithms are evaluated over five benchmark datasets. Experimental results with comparisons show that the DAC-based learning algorithm performs favorably in terms of effectiveness, efficiency and computational complexity, followed by the ADMM-based learning algorithm with promising accuracy but higher computational burden.

Over the last years, automatic music classification has become a standard benchmark problem in the machine learning community. This is partly due to its inherent difficulty, and also to the impact that a fully automated classification system can have in a commercial application. In this paper we test the efficiency of a relatively new learning tool, Extreme Learning Machines (ELM), for several classification tasks on publicly available song datasets. ELM is gaining increasing attention, due to its versatility and speed in adapting its internal parameters. Since both of these attributes are fundamental in music classification, ELM provides a good alternative to standard learning models. Our results support this claim, showing a sustained gain of ELM over a feedforward neural network architecture. In particular, ELM provides a great decrease in computational training time, and has always higher or comparable results in terms of efficiency.