Papers by Philippe Mulhem
Cet article decrit une contrainte d'un modele de recherche d'information decrivant les co... more Cet article decrit une contrainte d'un modele de recherche d'information decrivant les comportement attendu d'un systeme si un document du corpus est pose en requete, la contrainte DDMC (Document-Document Matching Constraint). Cette contrainte n'etant pas verifiee par un modele classique de recherche d'information (modele de langue base sur un calcul de nega-tive de Divergence de Kullback-Leibler avec lissage de Jelinek-Mercer), nous presentons une modification de ce dernier modele qui permet de verifier DDMC. Une derniere partie presente des experimentations menees afin de verifier que notre modification n'impacte pas la qualite des reponses d'un systeme, tout en garantissant la verification de DDMC.
Photographic images annotation is a complex problem. Indeed, the visual character- istics of obje... more Photographic images annotation is a complex problem. Indeed, the visual character- istics of objects of a class vary with the considered instance and the shooting conditions. In this paper we proposed a visual characterization of object parts, called "Visual Phrase", robust to these variations. A Visual Phrase is a set of regions of interest built according to pre-difined criteria; a topological criterium was studied in this paper. An automatic annotation method is proposed based on our definition and characterization of Visual Phrases. An experiment on VOC2009 corpus is presented, and we show that the fusion of our method with a standard bag of visual words approach on full images provides better results than those obtained via the standard approach. MOTS-CLES : Phrase Visuelle, Region d'interet, Sac de mot visuel, Annotation d'image
RESUME. Cet article decrit et defini l’utilisation de requetes par l’exemple (QBE) dans le cadre ... more RESUME. Cet article decrit et defini l’utilisation de requetes par l’exemple (QBE) dans le cadre de recherche symbolique d’images photographiques. La nouveaute de cette approche consiste en l’utilisation conjointe d’indexation symbolique automatique et d’un formalisme de representation de connaissances pour representer le contenu des images. De plus, le mecanisme d’abstraction perm la recherche d’images par l’exemple et le bouclage de pertinence bases sur la representation symbolique des images, et pas sur leur description signal de bas niveau. Nous montrons sur deux collections d’images d’un total de plus de 1100 photographies que la recherche par l’exemple fournit des resultats comparables a ceux par symboles en terme de mesures de rappel-precision.
This paper presents a novel approach, the first to our knowledge, tha t exploits a com- plete ext... more This paper presents a novel approach, the first to our knowledge, tha t exploits a com- plete extension of the language modeling approach from information retrieva l to the problem of graph-based image retrieval and categorization. Since photographic im ages are 2D data, we first use image regions and local interest points (mapped to automatically in duced concepts) and then relationships between these regions to build a complete graph repr esentation of im- ages. The results obtained on categorizing of RobotVision collection from Im ageCLEF 2009 (containing of 5 rooms in an indoor environment) show that (a) the proce dure to automatically induce concepts from an image is effective, and (b) the use of spatial relatio nships, in addition to concepts, for representing an image content helps improve the classifie r accuracy.
We describe here a method to use a graph language modeling approach fo image retrieval and image ... more We describe here a method to use a graph language modeling approach fo image retrieval and image categorization. Since photographic images are 2D da ta, we first use im- age regions (mapped to automatically induced concepts) and then spatial r elationships between
A Two-time Model for Video Content Representation and Retrieval
Electronic Workshops in Computing, 1999
Electronic Workshops in Computing, 1995
In this paper we give a technical description of the PRIME system prototype. PRIME allows us to p... more In this paper we give a technical description of the PRIME system prototype. PRIME allows us to provide an operational side to the theoretical work we have done in the "Modelling and Multimedia Information Retrieval" (MRIM) team. PRIME is designed to provide a generic way to express the storing, manipulating and retrieval of multimedia data. These tasks are separated into two parts, namely the strict database tasks and the information retrieval tasks. This paper focus on this generic part, and we address more specifically the problem of managing and retrieving images. We describe the implementation of a medical application managing Magnetic Resonance Images based on the generic core.
2016 14th International Workshop on Content-Based Multimedia Indexing (CBMI), 2016
This paper describes a method for querying lifelog data from visual content and from metadata ass... more This paper describes a method for querying lifelog data from visual content and from metadata associated with the recorded images. Our approach mainly relies on mapping the query terms to visual concepts computed on the Lifelogs images according to two separated learning schemes based on use of deep visual features. A post-processing is then performed if the topic is related to time, location or activity information associated with the images. This work was evaluated in the context of the Lifelog Semantic Access sub-task of the NTCIR-12 (2016). The results obtained are promising for a first participation to such a task, with an event-based MAP above 29% and an event-based nDCG value close to 39%.
International Conference on Multimedia and Expo, Jul 30, 2000
The extraction of objects present in photographs is a major problem to tackle when considering ph... more The extraction of objects present in photographs is a major problem to tackle when considering photographs retrieval. Such objects (or people) have to be detected in a way to allow retrieval based on concepts and not on physical characteristics like colors or textures. In a way to achieve a reasonable detection rate, in context image analysis has to be used. We propose here to use conceptual graphs (a knowledge representation formalism that allow fast processing) with Dempster-Shafer theory of evidence to update original labeling coming from a segmentation that labels image regions out of context. We explain the whole process and the results obtained on real home photos. The encouraging results obtained show the potential of our proposal.
LIG at TRECVid 2014: Semantic Indexing
PRIME is a precision-oriented information retrieval system, managing multimedia structured docume... more PRIME is a precision-oriented information retrieval system, managing multimedia structured documents. Such a system is primarily aimed to retrieve not necessary all, but mostly relevant documents. On the other hand, a precision-oriented system is also able to process precise queries corresponding to elaborated users' information needs. Such systems are based on complex and controled content representation languages fr documents, allowing a deep semantic understanding of the documents and the queries. The actual precision of an information retrieval system may be evaluated when instanciating it onto applications and a category of users. This paper describes the PRIME system, a direct experimentation of various models and approaches developed in the context of the FERMI project. The main concepts being implemented in this experiments are the combination of querying and browsing to improve retrieval performance, the consideration of structured information, the image and text models, and nally the use of conceptual graphs as a basic model for knowledge representation and manipulation. Two application corpuses are considered for achieving the experiments. The rst one is a medical corpus of documents combining medical reports and related sries of images in the domain of radiology. This rst corpus is mainly intendend to illustrate the combination of querying and browsing, and the notion of structured multimedia data (here combining texts and images). The second contains a collection of art images (photos) of various topics ranging from architecture, street scenery, landscapes to portraits and artifacts. The main goal here is to evaluate the e ectiveness of the image model when applied to intricate and heterogeneous image data. Both applications are based on the same PRIME platform, whose software architecture is built on the O2 object oriented database system. The interfaces are developed in X-Motif and HTML using forms.
The Quaero group is a consortium of French and German organizations working on Multimedia Indexin... more The Quaero group is a consortium of French and German organizations working on Multimedia Indexing and Retrieval 1. LIG participated to the semantic indexing main task, localization task and concept pair task. LIG also participated to the organization of this task. This paper describes these participations which are quite similar to our previous year's participations. For the semantic indexing main task, our approach uses a six-stages processing pipelines for computing scores for the likelihood of a video shot to contain a target concept. These scores are then used for producing a ranked list of images or shots that are the most likely to contain the target concept. The pipeline is composed of the following steps: descriptor extraction, descriptor optimization, classification, fusion of descriptor variants, higher-level fusion, and re-ranking. We used a number of different descriptors and a hierarchical fusion strategy. We also used conceptual feedback by adding a vector of classification score to the pool of descriptors. The best Quaero run has a Mean Inferred Average Precision of 0.2848, which ranked us 2 nd out of 26 participants. We also co-organized the TRECVid SIN 2013 task and collaborative annotation.
This working notes describe the runs and results obtained by the LIG at ImageCLEFphoto 2008. The ... more This working notes describe the runs and results obtained by the LIG at ImageCLEFphoto 2008. The submitted runs are: two runs (text only and text+image) without diversification on classes, and two runs (text only and text+image) with class diversification were submitted. The text retrieval is based on language model of Information Retrieval, and the image part is processed using RGB histograms on 9 image blocks with a similarity value based on Jeffrey divergence. Results using text+image are obtained by a linear combination of normalized results on text and image. The diversification is based on clusters, according to the cluster given in the queries. When the cluster name is not directly extracted from the images (like city or country), we apply a visual clustering. Not surprisingly, the cluster recall at 20 (i.e., cr(20)) results are higher for the runs that include diversification. On the other hand, the precision at 20 and the mean average precision results are higher without diversification on our runs, for both text only and image+text results.
This paper describes the different experiments that have been conducted by the MRIM group at the ... more This paper describes the different experiments that have been conducted by the MRIM group at the LIG in Grenoble for the ImageCLEF 2009 campaign. The group participated in the following tasks: Image Retrieval and Image Annotation. For the Image Retrieval task, we submitted runs with both text and image features, and a diversification process was applied. For the Image Annotation task, we used several features and classifiers in a way to generate keyword descriptions. For these two tasks, the results obtained are above the average of the participants.
This paper presents a component of a content based image retrieval system dedicated to let a user... more This paper presents a component of a content based image retrieval system dedicated to let a user defines the indexing terms used later during retrieval. A user inputs a indexing term name, image examples and counterexamples of the term, and the system learns a model of the concept as well as a similarity measure for this term. The similarity measure is based on weights reflecting the importance of each low-level feature extracted from the images. The system computes these weights using a genetic algorithm. Rating a particular similarity measure is done by clustering the examples and counterexamples using these weights and computing the quality of the obtained clusters. Experiments are conducted and results are presented on a set of 600 images.
Towards personalized image retrieval
Modèles pour résumés adaptatifs de vidéos
Ingénierie des systèmes d'information, 2002
Proceedings of the eleventh international conference on Information and knowledge management - CIKM '02, 2002
Photograph retrieval systems face the difficulty to deal with the different ways to apprehend the... more Photograph retrieval systems face the difficulty to deal with the different ways to apprehend the content of images. We consider and demonstrate here the use of multiple index representations of photographs to achieve effective retrieval. The use of multiple indexes allows integration of the complementary strengths of different indexing and retrieval models. The proposed representation supports multiple labels for regions and attributes, and handles inferences and relationships. We define links between indexing levels and the related query modes. The experiment conducted on 2400 home photographs shows the behavior of the multiple indexing levels during retrieval.
Modèle de graphe et modèle de langue pour la reconnaissance de scènes visuelles
Document numérique, 2010
Proceedings of CBMI, 2003
We present in this paper a relational approach for indexing and retrieving photographs from a col... more We present in this paper a relational approach for indexing and retrieving photographs from a collection. Instead of using simple keywords as an indexing language, we propose to use star-graphs as document descriptors. A star-graph is a conceptual graph that contains a single relation, with some concepts linked to it. They are elementary pieces of information describing combinations of concepts. We use stargraphs as descriptors-or index terms-for image content representation. This allows for relational indexing and expression ...
Uploads
Papers by Philippe Mulhem