Text classification with the support of pruned dependency patterns

In recent years, there has been an exponential growth in the number of complex documents and texts that require a deeper understanding of machine learning methods to be able to accurately classify texts in many applications. Many machine learning approaches have achieved surpassing results in natural language processing. The success of these learning algorithms relies on their capacity to understand complex models and non-linear relationships within data. However, finding suitable structures, architectures, and techniques for text classification is a challenge for researchers. In this paper, a brief overview of text classification algorithms is discussed. This overview covers different text feature extractions, dimensionality reduction methods, existing algorithms and techniques, and evaluations methods. Finally, the limitations of each technique and their application in real-world problems are discussed.

2018

Textual data is a high-dimensional data. In high-dimensional data, the number of features xceeds the number of samples. Hence, it equally increased the amount of noise, and irrelevant features. At this point, dimensionality reduction is necessary. Feature selection is an example of dimensionality reduction techniques. Moreover, it had been an indispensable component in classification. Thus, in this paper, we presented three feature selection approaches; filter, wrapper and embedded. Their aims, advantages and disadvantages are also briefly explained. Besides, this study reviews several significant studies for each feature selection approach for text classification. Based on the studies, we found that wrapper approach is less used by researchers since it is prone to over-fit and exposed local-optima for text classification while filter and embedded achieved an amount of research. However, in filter approach, the classification accuracies cannot be guaranteed because it does not incor...

KSII Transactions on Internet and Information Systems, 2019

The performance of text classification is highly related to the feature selection methods. Usually, two tasks are performed when a feature selection method is applied to construct a feature set; 1) assign score to each feature and 2) select the top-N features. The selection of top-N features in the existing filter-based feature selection methods is biased by their discriminative power and the empirical process which is followed to determine the value of N. In order to improve the text classification performance by presenting a more illustrative feature set, we present an approach via a potent representation learning technique, namely DBN (Deep Belief Network). This algorithm learns via the semantic illustration of documents and uses feature vectors for their formulation. The nodes, iteration, and a number of hidden layers are the main parameters of DBN, which can tune to improve the classifier's performance. The results of experiments indicate the effectiveness of the proposed method to increase the classification performance and aid developers to make effective decisions in certain domains.

Web Intelligence, 2020

Text classification (a.k.a text categorisation) is an effective and efficient technology for information organisation and management. With the explosion of information resources on the Web and corporate intranets continues to increase, it has being become more and more important and has attracted wide attention from many different research fields. In the literature, many feature selection methods and classification algorithms have been proposed. It also has important applications in the real world. However, the dramatic increase in the availability of massive text data from various sources is creating a number of issues and challenges for text classification such as scalability issues. The purpose of this report is to give an overview of existing text classification technologies for building more reliable text classification applications, to propose a research direction for addressing the challenging problems in text mining.

This paper proposes a method for Feature Selection in Text Categorization. This task is performed in two steps. Firstly, an analysis of relevance is performed and after that analysis of redundancy is done. For this purpose, a range of similarity measures are adopted and converted into symmetrical ones using several aggregation operators. This fact assures that the similarity between two words are independent of the order they are considered. Several experiments over four corpora are performed, leading to conclude that this method reaches good results.

International Journal of Control and Automation, 2019

Text classification for data preprocessing methods regularly uses bagof-words (BoWs). In a large dataset, BoWs always include many vectors with very large sizes and high dimensions. The authors introduced a new data preprocessing method for feature reduction of short text classification, namely NDTMD. It reduces features of the dataset using BoWs and word embedding (WE), and can solve the weaknesses of BoWs. The experiment consisted of four steps: 1) 5 datasets were selected from the data science community website, Kaggle; 2) the new methods were compared with 5 commonly used data preprocessing methods and 4 of these 5 methods used the state of the art as their baseline, while the other one used BoWs. One of the new data preprocessing methods used features reduction of BoWs to produce a new document termed matrix data (NDTMD); 3) the authors generated many classification models by 3 classifiers: support vector machine, logistic regression, and convolutional neural network for text classification; and 4) the above classifiers were applied to each preprocessing dataset and evaluated using feature reduction rate (FRR), accuracy, kappa, and running time performance. The results showed that classification models had the highest performance when using NDTMD. In particular, classifier algorithms had the highest accuracy and kappa but the lowest running time. The new data preprocessing methods can be used to preprocess short text classification and also can be applied with real social media data.

arXiv (Cornell University), 2021

Text classification is a fundamental problem in the field of natural language processing. Text classification mainly focuses on giving more importance to all the relevant features that help classify the textual data. Apart from these, the text can have redundant or highly correlated features. These features increase the complexity of the classification algorithm. Thus, many dimensionality reduction methods were proposed with the traditional machine learning classifiers. The use of dimensionality reduction methods with machine learning classifiers has achieved good results. In this paper, we propose a hybrid feature selection method for obtaining relevant features by combining various filter-based feature selection methods and fastText classifier. We then present three ways of implementing a feature selection and neural network pipeline. We observed a reduction in training time when feature selection methods are used along with neural networks. We also observed a slight increase in accuracy on some datasets.

1999

Most research in text classification has used the "bag of words" representation of text. This paper examines some alternative ways to represent text based on syntactic and semantic relationships between words (phrases, synonyms and hypernyms). We describe the new representations and try to justify our suspicions that they could have improved the performance of a rule-based learner. The representations are evaluated using the RIPPER rule-based learner on the Reuters-21578 and DigiTrad test corpora, but on their own the new representations are not found to produce a significant performance improvement. Finally, we try combining classifiers based on different representations using a majority voting technique. This step does produce some performance improvement on both test collections. In general, our work supports the emerging consensus in the information retrieval community that more sophisticated Natural Language Processing techniques need to be developed before better text representations can be produced. We conclude that for now, research into new learning algorithms and methods for combining existing learners holds the most promise.

2002

Recently, an original extension of the well-known Rocchio model (i.e. the Generalized Rocchio Formula (¡ £¢ ¥¤ )) as a feature weighting method for text classification has been presented. The assessment of such a model requires a statistically motivated parameter estimation method and wider empirical evidence. In this paper, three different corpora have been adopted in two languages. Results suggest that ¡ £¢ £¤ , integrating linguistic information, is a viable more efficient alternative to state-of-art ¦ §¤ systems. ¤ © ¢¡ ¤£ ¥£ ¥£ ¥£ ¥¡ ¤ §¦ ©¨, representing topics (e.g. "Politics, Entertainment"). An extensive collection of texts already classified, often called training set, induces the classification function. Profile-based (or linear) classifiers are characterized by a function based on a similarity measure between the representation of incoming documents and each class ¤ . Both representations are vectors and similarity is traditionally estimated as the cosine angle between the two vectors. The description

A simple and efficient baseline for text classification is to represent sentences as bag-of-words (BoW) and train a linear classifier. The bag-of-words model is simple to implement and offers flexibility for customization by providing different scoring techniques for user specific text data. In many problem domains, these linear classifiers are preferred over more complex models like CNN, LSTM because of their efficiency, robustness and interpretability. However, a large vocabulary can cause extremely sparse representations which are harder to model, where the challenge is for models to harness very little information in such a large representational space. Also, these classification problems are categorized by large number of classes and highly imbalanced distribution of data across these classes. In such cases, the traditional linear classifiers would treat each word separately and assign them different coefficients based on the frequency in which they occur in the train set. This would result in lower test accuracy when it comes across instances where a word which was occurring less frequently in the train set, occurs more often in the test set. Our thesis aims to solve this problem by constraining weights of rare features by similar, more frequent ones, using semantic similarity. This would enforce similar words to have similar weights thereby improving model performance. Thus, based on how similar two features are, our proposed model can improve the feature importance of a sparse word by increasing its regression coefficient , thereby improving the test accuracy in the above mentioned scenario.