Learning multi-label scene classification

In classic pattern recognition problems, classes are mutually exclusive by deÿnition. Classiÿcation errors occur when the classes overlap in the feature space. We examine a di erent situation, occurring when the classes are, by deÿnition, not mutually exclusive. Such problems arise in semantic scene and document classiÿcation and in medical diagnosis. We present a framework to handle such problems and apply it to the problem of semantic scene classiÿcation, where a natural scene may contain multiple objects such that the scene can be described by multiple class labels (e.g., a ÿeld scene with a mountain in the background). Such a problem poses challenges to the classic pattern recognition paradigm and demands a di erent treatment. We discuss approaches for training and testing in this scenario and introduce new metrics for evaluating individual examples, class recall and precision, and overall accuracy. Experiments show that our methods are suitable for scene classiÿcation; furthermore, our work appears to generalize to other classiÿcation problems of the same nature.

2003

In classic pattern recognition problems, classes are mutually exclusive by denition. Classication errors occur when the classes overlap in the feature space. We examine a dieren t situation, occurring when the classes are, by denition, not mutually exclusive. Such problems arise in semantic scene and document classication and in medical diagnosis. We present a framework to handle such problems

Storage and Retrieval Methods and Applications for Multimedia 2004, 2003

In classic pattern recognition problems, classes are mutually exclusive by definition. Classification errors occur when the classes overlap in the feature space. We examine a different situation, occurring when the classes are, by definition, not mutually exclusive. Such problems arise in scene and document classification and in medical diagnosis. We present a framework to handle such problems and apply it to the problem of semantic scene classification, where a natural scene may contain multiple objects such that the scene can be described by multiple class labels (e.g., a field scene with a mountain in the background). Such a problem poses challenges to the classic pattern recognition paradigm and demands a different treatment. We discuss approaches for training and testing in this scenario and introduce new metrics for evaluating individual examples, class recall and precision, and overall accuracy. Experiments show that our methods are suitable for scene classification; furthermore, our work appears to generalize to other classification problems of the same nature.

2010

In multi-label learning, an image containing multiple objects can be assigned to multiple labels, which makes it more challenging than traditional multi-class classification task where an image is assigned to only one label. In this paper, we propose a multi-label learning framework based on Imageto-Class (I2C) distance, which is recently shown useful for image classification. We adjust this I2C distance to cater for the multi-label problem by learning a weight attached to each local feature patch and formulating it into a large margin optimization problem. For each image, we constrain its weighted I2C distance to the relevant class to be much less than its distance to other irrelevant class, by the use of a margin in the optimization problem. Label ranks are generated under this learned I2C distance framework for a query image. Thereafter, we employ the label correlation information to split the label rank for predicting the label(s) of this query image. The proposed method is evaluated in the applications of scene classification and automatic image annotation using both the natural scene dataset and Microsoft Research Cambridge (MSRC) dataset. Experiment results show better performance of our method compared to previous multi-label learning algorithms.

Multi-instance multi-label learning (MIML) is a framework for learning in the presence of label ambiguity. In MIML, experts provide labels for groups of instances (bags), instead of directly providing a label for every instance. When labeling efforts are focused on a set of target classes, instances outside this set will not be appropriately modeled. For example, ornithologists label bird audio recordings with a list of species present. Other additional sound instances, e.g., a rain drop or a moving vehicle sound, are not labeled. The challenge is due to the fact that for a given bag, the presence or absence of novel instances is latent. In this paper, this problem is addressed using a discriminative probabilistic model that accounts for novel instances. We propose an exact and efficient implementation of the maximum likelihood approach to determine the model parameters and consequently learn an instance-level classifier for all classes including the novel class. Experiments on both synthetic and real datasets illustrate the effectiveness of the proposed approach.

Abstract This paper presents a simple and general approach for multi-label problems. In contrast with the commonly used binary approach, which roots from the 1-vs-rest strategy for multi-class classification, the new method extends the 1-vs-1 strategy and conducts pairwise training. While the binary approach assumes labels are assigned independently, the proposed method better incorporates information between any two labels.

ArXiv, 2022

With increasing applications of semantic segmentation, numerous datasets have been proposed in the past few years. Yet labeling remains expensive, thus, it is desirable to jointly train models across aggregations of datasets to enhance data volume and diversity. However, label spaces differ across datasets and may even be in conflict with one another. This paper proposes UniSeg, an effective approach to automatically train models across multiple datasets with differing label spaces, without any manual relabeling efforts. Specifically, we propose two losses that account for conflicting and co-occurring labels to achieve better generalization performance in unseen domains. First, a gradient conflict in training due to mismatched label spaces is identified and a class-independent binary cross-entropy loss is proposed to alleviate such label conflicts. Second, a loss function that considers class-relationships across datasets is proposed for a better multi-dataset training scheme. Exten...

2006 18th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'06), 2006

In classic pattern recognition problems, classes are mutually exclusive by definition. However, in many applications, it is quite natural that some instances belong to multiple classes at the same time. In other words, these applications are multi-labeled, classes are overlapped by definition and each instance may be associated to multiple classes. In this paper, we present a comparative study on various multi-label approaches using both gene and scene data sets.

Multiple Instance Multiple Label learning problem has received much attention in machine learning and computer vision literature due to its applications in image clas- sification and object detection. However, the current state-of-the-art solutions to this problem lack scalability and cannot be applied to datasets with a large number of in- stances and a large number of labels. In this paper we present a novel learning algorithm for Multiple Instance Multiple Label learning that is scalable for large datasets and per- forms comparable to the state-of-the-art algorithms. The proposed algorithm trains a set of discriminative multiple instance classifiers (one for each label in the vocabulary of all possible labels) and models the correlations among labels by finding a low rank weight matrix thus forcing the classifiers to share weights. This algorithm is a linear model un- like the state-of-the-art kernel methods which need to compute the kernel matrix. The model parameters are efficiently learned by solving an unconstrained optimization prob- lem for which Stochastic Gradient Descent can be used to avoid storing all the data in memory.

ArXiv, 2021

Transfer learning from large-scale pre-trained models has become essential for many computer vision tasks. Recent studies have shown that datasets like ImageNet are weakly labeled since images with multiple object classes present are assigned a single label. This ambiguity biases models towards a single prediction, which could result in the suppression of classes that tend to co-occur in the data. Inspired by language emergence literature, we propose multi-label iterated learning (MILe) to incorporate the inductive biases of multi-label learning from single labels using the framework of iterated learning. MILe is a simple yet effective procedure that builds a multi-label description of the image by propagating binary predictions through successive generations of teacher and student networks with a learning bottleneck. Experiments show that our approach exhibits systematic benefits on ImageNet accuracy as well as ReaL F1 score, which indicates that MILe deals better with label ambigu...