Feature Selection for Classification with QAOA
Maurizio Ferrari Dacrema2022, arXiv (Cornell University)
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
AI
AI
Feature selection is of great importance in Machine Learning, where it can be used to reduce the dimensionality of classification, ranking and prediction problems. The removal of redundant and noisy features can improve both the accuracy and scalability of the trained models. However, feature selection is a computationally expensive task with a solution space that grows combinatorically. In this work, we consider in particular a quadratic feature selection problem that can be tackled with the Quantum Approximate Optimization Algorithm (QAOA), already employed in combinatorial optimization. First we represent the feature selection problem with the QUBO formulation, which is then mapped to an Ising spin Hamiltonian. Then we apply QAOA with the goal of finding the ground state of this Hamiltonian, which corresponds to the optimal selection of features. In our experiments, we consider seven different real-world datasets with dimensionality up to 21 and run QAOA on both a quantum simulator and, for small datasets, the 7-qubit IBM (ibm-perth) quantum computer. We use the set of selected features to train a classification model and evaluate its accuracy. Our analysis shows that it is possible to tackle the feature selection problem with QAOA and that currently available quantum devices can be used effectively. Future studies could test a wider range of classification models as well as improve the effectiveness of QAOA by exploring better performing optimizers for its classical step.
Related papers
Towards Feature Selection for Ranking and Classification Exploiting Quantum Annealers
Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
Feature selection is a common step in many ranking, classification, or prediction tasks and serves many purposes. By removing redundant or noisy features, the accuracy of ranking or classification can be improved and the computational cost of the subsequent learning steps can be reduced. However, feature selection can be itself a computationally expensive process. While for decades confined to theoretical algorithmic papers, quantum computing is now becoming a viable tool to tackle realistic problems, in particular special-purpose solvers based on the Quantum Annealing paradigm. This paper aims to explore the feasibility of using currently available quantum computing architectures to solve some quadratic feature selection algorithms for both ranking and classification. The experimental analysis includes 15 state-of-the-art datasets. The effectiveness obtained with quantum computing hardware is comparable to that of classical solvers, indicating that quantum computers are now reliable enough to tackle interesting problems. In terms of scalability, current generation quantum computers are able to provide a limited speedup over certain classical algorithms and hybrid quantum-classical strategies show lower computational cost for problems of more than a thousand features. CCS CONCEPTS • Information systems → Content analysis and feature selection; • Computer systems organization → Quantum computing.
A hybrid quantum feature selection algorithm using a quantum inspired graph theoretic approach
Applied Intelligence, 2020
Quantum machine learning bridges the gap between abstract developments in quantum computing and the applied research on machine learning. It generally exposes the synthesis of important machine learning algorithms in a quantum framework. Dimensionality reduction of a dataset with a suitable feature selection strategy is one of the most important tasks in knowledge discovery and data mining. The efficient feature selection strategy helps to improve the overall accuracy of a large dataset in terms of machine learning operations. In this paper, a quantum feature selection algorithm using a graph-theoretic approach has been proposed. The proposed algorithm has used the concept of correlation coefficient based graph-theoretic classical approach initially and then applied the quantum Oracle with CNOT operation to verify whether the dataset is suitable for dimensionality reduction or not. If it is suitable, then our algorithm can efficiently estimate their high correlation values by using quantum parallel amplitude estimation and amplitude amplification techniques. This paper also shows that our proposed algorithm substantially outperforms than some popular classical feature selection algorithms for supervised classification in terms of query complexity of O(k N (k) c N (k) f), where N is the size of the feature vectors whose values are T H min (minimum threshold), k is the number of iterations and where is the error for estimating those feature vectors. Compared with the classical counterpart, i.e. the performance of our quantum algorithm quadratically improves than others.
Best Subset Selection: Statistical Computing Meets Quantum Computing
arXiv (Cornell University), 2021
With the rapid development of quantum computers, quantum algorithms have been studied extensively. However, quantum algorithms tackling statistical problems are still lacking. In this paper, we propose a novel non-oracular quantum adaptive search (QAS) method for the best subset selection problems. QAS performs almost identically to the naive best subset selection method but reduces its computational complexity from OpDq to Op ? D log 2 Dq, where D " 2 p is the total number of subsets over p covariates. Unlike existing quantum search algorithms, QAS does not require the oracle information of the true solution state and hence is applicable to various statistical learning problems with random observations. Theoretically, we prove QAS attains any arbitrary success probability q P p0.5, 1q within Oplog 2 Dq iterations. When the underlying regression model is linear, we propose a quantum linear prediction method that is faster than its classical counterpart. We further introduce a hybrid quantum-classical strategy to avoid the capacity bottleneck of existing quantum computing systems and boost the success probability of QAS by majority voting. The effectiveness of this strategy is justified by both theoretical analysis and extensive empirical experiments on quantum and classical computers.
Binary classifiers for noisy datasets: a comparative study of existing quantum machine learning frameworks and some new approaches
2022
One of the most promising areas of research to obtain practical advantage is Quantum Machine Learning which was born as a result of cross-fertilisation of ideas between Quantum Computing and Classical Machine Learning. In this paper, we apply Quantum Machine Learning (QML) frameworks to improve binary classification models for noisy datasets which are prevalent in financial datasets. The metric we use for assessing the performance of our quantum classifiers is the area under the receiver operating characteristic curve (ROC/AUC). By combining such approaches as hybrid-neural networks, parametric circuits, and data re-uploading we create QML inspired architectures and utilise them for the classification of non-convex 2 and 3-dimensional figures. An extensive benchmarking of our new FULL HYBRID classifiers against existing quantum and classical classifier models, reveals that our novel models exhibit better learning characteristics to asymmetrical Gaussian noise in the dataset compared...
A Quantum-inspired Version of the Classification Problem
International Journal of Theoretical Physics, 2017
We address the problem of binary classification by using a quantum version of the Nearest Mean Classifier (NMC). Our proposal is indeed an advanced version of previous one (see Sergioli et al. 2017 that i) is able to be naturally generalized to arbitrary number of features and ii) exhibits better performances with respect to the classical NMC for several datasets. Further, we show that the quantum version of NMC is not invariant under rescaling. This allows us to introduce a free parameter, i.e. the rescaling factor, that could be useful to get a further improvement of the classification performance.
Image Classification via Quantum Machine Learning
ArXiv, 2020
Quantum Computing and especially Quantum Machine Learning, in a short period of time, has gained a lot of interest through research groups around the world. This can be seen in the increasing number of proposed models for pattern classification applying quantum principles to a certain degree. Despise the increasing volume of models, there is a void in testing these models on real datasets and not only on synthetic ones. The objective of this work is to classify patterns with binary attributes using a quantum classifier. Specially, we show results of a complete quantum classifier applied to image datasets. The experiments show favorable output while dealing with balanced classification problems as well as with imbalanced classes where the minority class is the most relevant. This is promising in medical areas, where usually the important class is also the minority class.
A Review of Machine Learning Classification Using Quantum Annealing for Real-World Applications
SN Computer Science, 2021
Optimizing the training of a machine learning pipeline helps in reducing training costs and improving model performance. One such optimizing strategy is quantum annealing, which is an emerging computing paradigm that has shown potential in optimizing the training of a machine learning model. The implementation of a physical quantum annealer has been realized by D-wave systems and is available to the research community for experiments. Recent experimental results on a variety of machine learning applications using quantum annealing have shown interesting results where the performance of classical machine learning techniques is limited by limited training data and high dimensional features. This article explores the application of D-wave's quantum annealer for optimizing machine learning pipelines for real-world classification problems. We review the application domains on which a physical quantum annealer has been used to train machine learning classifiers. We discuss and analyze the experiments performed on the D-Wave quantum annealer for applications such as image recognition, remote sensing imagery, computational biology, and particle physics. We discuss the possible advantages and the problems for which quantum annealing is likely to be advantageous over classical computation.
Feature Selection for Recommender Systems with Quantum Computing
Entropy, 2021
The promise of quantum computing to open new unexplored possibilities in several scientific fields has been long discussed, but until recently the lack of a functional quantum computer has confined this discussion mostly to theoretical algorithmic papers. It was only in the last few years that small but functional quantum computers have become available to the broader research community. One paradigm in particular, quantum annealing, can be used to sample optimal solutions for a number of NP-hard optimization problems represented with classical operations research tools, providing an easy access to the potential of this emerging technology. One of the tasks that most naturally fits in this mathematical formulation is feature selection. In this paper, we investigate how to design a hybrid feature selection algorithm for recommender systems that leverages the domain knowledge and behavior hidden in the user interactions data. We represent the feature selection as an optimization probl...
Supervised Learning Using Quantum Technology
2020
In this paper, we present classical machine learning algorithms enhanced by quantum technology to classify a data set. The data set contains binary input variables and binary output variables. The goal is to extend classical approaches such as neural networks by using quantum machine learning principles. Classical algorithms struggle as the dimensionality of the feature space increases. We examine the usage of quantum technologies to speed up these classical algorithms and to introduce the new quantum paradigm into machine diagnostic domain. Most of the prognosis models based on binary or multi-valued classification have become increasingly complex throughout the recent past. Starting with a short introduction into quantum computing, we will present an approach of combining quantum computing and classical machine learning to the reader. Further, we show different implementations of quantum classification algorithms. Finally, the algorithms presented are applied to a data set. The re...
Related papers
Quantum-Classical Hybrid Machine Learning for Image Classification (ICCAD Special Session Paper)
2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD), 2021
Image classification is a major application domain for conventional deep learning (DL). Quantum machine learning (QML) has the potential to revolutionize image classification. In any typical DL-based image classification, we use convolutional neural network (CNN) to extract features from the image and multi-layer perceptron network (MLP) to create the actual decision boundaries. QML models can be useful in both of these tasks. On one hand, convolution with parameterized quantum circuits (Quanvolution) can extract rich features from the images. On the other hand, quantum neural network (QNN) models can create complex decision boundaries. Therefore, Quanvolution and QNN can be used to create an end-to-end QML model for image classification. Alternatively, we can extract image features separately using classical dimension reduction techniques such as, Principal Components Analysis (PCA) or Convolutional Autoencoder (CAE) and use the extracted features to train a QNN. We review two proposals on quantum-classical hybrid ML models for image classification namely, Quanvolutional Neural Network and dimension reduction using a classical algorithm followed by QNN. Particularly, we make a case for trainable filters in Quanvolution and CAE-based feature extraction for image datasets (instead of dimension reduction using linear transformations such as, PCA). We discuss various design choices, potential opportunities, and drawbacks of these models. We also release a Python-based framework to create and explore these hybrid models with a variety of design choices.
Study of Feature Importance for Quantum Machine Learning Models
2022
Feature or predictor importance is a crucial part of data preprocessing pipelines in classical machine learning. Since classical data is used in many quantum machine learning models, feature importance is equally important for quantum machine learning (QML) models. This work presents the first study of its kind in which feature importance for QML models has been explored and contrasted with their classical machine learning (CML) equivalents. We developed a hybrid quantum classical architecture where QML models are trained and feature importance values are calculated from classical algorithms on a realworld dataset. This architecture has been implemented on ESPN fantasy football data using Qiskit statevector simulators and IBM Quantum hardware such as the IBMQ Mumbai and IBMQ Montreal systems. Even though we are in the Noisy Intermediate-Scale Quantum (NISQ) era, the physical quantum computing results are promising. To facilitate current quantum scale, we created a data tiering, mode...
Project Report-Quantum Machine Learning
2017
The aim of the project is to study two of the most widely used machine learning strategies, namely KNearest Neighbours algorithm and Perceptron Learning algorithm, in a quantum setting, and study the speedups that the quantum modules allow over the classical counterparts. The study is primarily based on the following 3 papers: 1. Quantum Perceptron Models, by N. Wiebe, A. Kapoor and K. M. Svore. 2. Quantum Algorithm for K-Nearest Neighbors Classification Based on the Metric of Hamming Distance, by Y. Ruan, X. Xue, H. Liu, J. Tan, and X. Li. 3. Quantum Algorithms for Nearest-Neighbor Methods for Supervised and Unsupervised Learning, by N. Wiebe, A. Kapoor and K. M. Svore.
Performance Analysis of Quantum Classifier on Benchmarking Datasets
IJEER, 2022
Quantum machine learning (QML) is an evolving field which is capable of surpassing the classical machine learning in solving classification and clustering problems. The enormous growth in data size started creating barrier for classical machine learning techniques. QML stand out as a best solution to handle big and complex data. In this paper quantum support vector machine (QSVM) based models for the classification of three benchmarking datasets namely, Iris species, Pumpkin seed and Raisin has been constructed. These QSVM based classification models are implemented on real-time superconducting quantum computers/simulators. The performance of these classification models is evaluated in the context of execution time and accuracy and compared with the classical support vector machine (SVM) based models. The kernel based QSVM models for the classification of datasets when run on IBMQ_QASM_simulator appeared to be 232, 207 and 186 times faster than the SVM based classification model. The results indicate that quantum computers/algorithms deliver quantum speed-up.
ICCAD Special Session Paper: Quantum-Classical Hybrid Machine Learning for Image Classification
2021
Image classification is a major application domain for conventional deep learning (DL). Quantum machine learning (QML) has the potential to revolutionize image classification. In any typical DL-based image classification, we use convolutional neural network (CNN) to extract features from the image and multi-layer perceptron network (MLP) to create the actual decision boundaries. On one hand, QML models can be useful in both of these tasks. Convolution with parameterized quantum circuits (Quanvolution) can extract rich features from the images. On the other hand, quantum neural network (QNN) models can create complex decision boundaries. Therefore, Quanvolution and QNN can be used to create an end-to-end QML model for image classification. Alternatively, we can extract image features separately using classical dimension reduction techniques such as, Principal Components Analysis (PCA) or Convolutional Autoencoder (CAE) and use the extracted features to train a QNN. We review two prop...
Variational quantum algorithm for unconstrained black box binary optimization: Application to feature selection
Cornell University - arXiv, 2022
We introduce a variational quantum algorithm to solve unconstrained black box binary optimization problems, i.e., problems in which the objective function is given as black box. This is in contrast to the typical setting of quantum algorithms for optimization where a classical objective function is provided as a given Quadratic Unconstrained Binary Optimization problem and mapped to a sum of Pauli operators. Furthermore, we provide theoretical justification for our method based on convergence guarantees of quantum imaginary time evolution. To investigate the performance of our algorithm and its potential advantages, we tackle a challenging real-world optimization problem: feature selection. This refers to the problem of selecting a subset of relevant features to use for constructing a predictive model such as fraud detection. Optimal feature selection-when formulated in terms of a generic loss function-offers little structure on which to build classical heuristics, thus resulting primarily in 'greedy methods'. This leaves room for (near-term) quantum algorithms to be competitive to classical state-of-the-art approaches. We apply our quantum-optimization-based feature selection algorithm, termed VarQFS, to build a predictive model for a credit risk data set with 20 and 59 input features (qubits) and train the model using quantum hardware and tensor-network-based numerical simulations, respectively. We show that the quantum method produces competitive and in certain aspects even better performance compared to traditional feature selection techniques used in today's industry.
Quantum Ensemble for Classification
ArXiv, 2020
A powerful way to improve performance in machine learning is to construct an ensemble that combines the predictions of multiple models. Ensemble methods are often much more accurate and lower variance than the individual classifiers that make them up but have high requirements in terms of memory and computational time. In fact, a large number of alternative algorithms is usually adopted, each requiring to query all available data. We propose a new quantum algorithm that exploits quantum superposition, entanglement and interference to build an ensemble of classification models. Thanks to the generation of the several quantum trajectories in superposition, we obtain $B$ transformations of the quantum state which encodes the training set in only $log\left(B\right)$ operations. This implies an exponential speed-up in the size of the ensemble with respect to classical methods. Furthermore, when considering the overall cost of the algorithm, we show that the training of a single weak clas...
Quantum algorithms for supervised and unsupervised machine learning
Machine-learning tasks frequently involve problems of manipulating and classifying large numbers of vectors in high-dimensional spaces. Classical algorithms for solving such problems typically take time polynomial in the number of vectors and the dimension of the space. Quantum computers are good at manipulating high-dimensional vectors in large tensor product spaces. This paper provides supervised and unsupervised quantum machine learning algorithms for cluster assignment and cluster finding. Quantum machine learning can take time logarithmic in both the number of vectors and their dimension, an exponential speed-up over classical algorithms. In machine learning, information processors perform tasks of sorting, assembling, assimilating, and classifying information [1-2]. In supervised learning, the machine infers a function from a set of training examples. In unsupervised learning the machine tries to find hidden structure in unlabeled data. Recent studies and applications focus in particular on the problem of large-scale machine learning [2]-big data-where the training set and/or the number of features is large. Various results on quantum machine learning investigate the use of quantum information processors to perform machine learning tasks [3-9], including pattern matching [3], Probably Approximately Correct learning [4], feedback learning for quantum measurement [5], binary classifiers [6-7], and quantum support vector machines [8].
Quantum enhanced machine learning: An overview
Machine learning is now widely used almost everywhere, primarily for forecasting. The main idea of the work is to identify the possibility of achieving a quantum advantage when solving machine learning problems on a quantum computer.
Quantum support vector machine for big data classification
Physical review letters, 2014
Supervised machine learning is the classification of new data based on already classified training examples. In this work, we show that the support vector machine, an optimized binary classifier, can be implemented on a quantum computer, with complexity logarithmic in the size of the vectors and the number of training examples. In cases where classical sampling algorithms require polynomial time, an exponential speedup is obtained. At the core of this quantum big data algorithm is a nonsparse matrix exponentiation technique for efficiently performing a matrix inversion of the training data inner-product (kernel) matrix.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.