A Novel Framework for Video Search

International Journal of Multimedia Information Retrieval, 2015

Semantic search or text-to-video search in video is a novel and challenging problem in information and multimedia retrieval. Existing solutions are mainly limited to text-to-text matching, in which the query words are matched against the user-generated metadata. This kind of text-to-text search, though simple, is of limited functionality as it provides no understanding about the video content. This paper presents a state-of-the-art system for event search without any user-generated metadata or example videos, known as text-to-video search. The system relies on substantial video content understanding and allows for searching complex events over a large collection of videos. The proposed text-tovideo search can be used to augment the existing text-to-text search for video. The novelty and practicality are demonstrated by the evaluation in NIST TRECVID 2014, where the proposed system achieves the best performance. We share our observations and lessons in building such a state-of-theart system, which may be instrumental in guiding the design of the future system for video search and analysis.

Proceedings - 14th International Conference on Image Analysis and Processing Workshops, ICIAP 2007, 2007

In this paper we describe the current performance of our MediaMill system as presented in the TRECVID 2006 benchmark for video search engines. The MediaMill team participated in two tasks: concept detection and search. For concept detection we use the MediaMill Challenge as experimental platform. The MediaMill Challenge divides the generic video indexing problem into a visual-only, textualonly, early fusion, late fusion, and combined analysis experiment. We provide a baseline implementation for each experiment together with baseline results. We extract image features, on global, regional, and keypoint level, which we combine with various supervised learners. A late fusion approach of visual-only analysis methods using geometric mean was our most successful run. With this run we conquer the Challenge baseline by more than 50%. Our concept detection experiments have resulted in the best score for three concepts: i.e. desert, flag us, and charts. What is more, using LSCOM annotations, our visual-only approach generalizes well to a set of 491 concept detectors. To handle such a large thesaurus in retrieval, an engine is developed which allows users to select relevant concept detectors based on interactive browsing using advanced visualizations. Similar to previous years our best interactive search runs yield top performance, ranking 2nd and 6th overall.

Citation/Export MLA Nikhil Dhonge, Akshay Kumbhare, Nikesh Dudhe, Kalyani Satone, “A Survey on Video Recommendation and Ranking in Video Search Engine”, February 15 Volume 3 Issue 2 , International Journal on Recent and Innovation Trends in Computing and Communication (IJRITCC), ISSN: 2321-8169, PP: 504 - 507, DOI: 10.17762/ijritcc2321-8169.150216 APA Nikhil Dhonge, Akshay Kumbhare, Nikesh Dudhe, Kalyani Satone, February 15 Volume 3 Issue 2, “A Survey on Video Recommendation and Ranking in Video Search Engine”, International Journal on Recent and Innovation Trends in Computing and Communication (IJRITCC), ISSN: 2321-8169, PP: 504 - 507, DOI: 10.17762/ijritcc2321-8169.150216

J Amer Coll Cardiol, 2011

In this paper we describe our TRECVID 2010 video retrieval experiments. The MediaMill team participated in three tasks: semantic indexing, known-item search, and instance search. The starting point for the MediaMill concept detection approach is our top-performing bag-of-words system of TRECVID 2009, which uses multiple color SIFT descriptors, sparse codebooks with spatial pyramids, kernelbased machine learning, and multi-frame video processing. We improve upon this baseline system by further speeding up its execution times for both training and classification using GPU-optimized algorithms, approximated histogram intersection kernels, and several multi-frame combination methods. Being more efficient allowed us to supplement the Internet video training collection with positively labeled examples from international news broadcasts and Dutch documentary video from the TRECVID 2005-2009 benchmarks. Our experimental setup covered a huge training set of 170 thousand keyframes and a test set of 600 thousand keyframes in total. Ultimately leading to 130 robust concept detectors for video retrieval. For retrieval, a robust but limited set of concept detectors justifies the need to rely on as many auxiliary information channels as possible. For automatic known item search we therefore explore how we can learn to rank various information channels simultaneously to maximize video search results for a given topic. To further improve the video retrieval results, our interactive known item search experiments investigate how to combine metadata search and visualization into a single interface. The 2010 edition of the TRECVID benchmark has again been a fruitful participation for the MediaMill team, resulting in the top ranking for concept detection in the semantic indexing task. Feature Extraction Feature Extraction Word Projection Word Projection Machine Learning Machine Learning

2019

In this paper, we describe the systems developed for Ad-hoc Video Search (AVS) task at TRECVID 2019[1] and the achieved results. Ad-Hoc Video Search (AVS): We merge three video search systems for AVS, including: two conceptbased video search systems which analyse the query using linguistic approaches then select and fuse the concepts, and a video retrieval model which learns the joint embedding space of the textual queries and the videos for matching. With this setting, we plan to analyze the advantages and shortcomings of these video search approaches. We submit totally seven runs consisting four automatic runs, two manual runs, and one novelty run. We brief our runs as follows: • F_M_C_D_VIREO.19_1 : This automatic run has mean xinfAP=0.034 using a concept-based video search system including ∼16.6k concepts covering objects, persons, activities, and places. We parse the queries with Stanford NLP parsing tool [2], keep the keywords, and categorize the keywords into three groups: object/person, action, and place. Correspondingly, the concepts from different groups in the concept bank are selected and fused.

ITE Transactions on Media Technology and Applications, 2016

Shoou-I Yu* (student member) †1 , Yi Yang (member) †2 , Zhongwen Xu (student member) †2 , Shicheng Xu (student member) †1 , Deyu Meng (member) †3 , Zexi Mao (member) †1 , Zhigang Ma (member) †1 , Ming Lin (member) †1 , Xuanchong Li (student member) †1 , Huan Li (member) †1 , Zhenzhong Lan (student member) †1 , Lu Jiang (student member) †1 , Alexander G. Hauptmann (member) †1 , Chuang Gan (student member) †4 , Xingzhong Du (student member) †5 , Xiaojun Chang (student member) †2

2010

In this paper we describe our TRECVID 2009 video retrieval experiments. The MediaMill team participated in three tasks: concept detection, automatic search, and interactive search. Starting point for the MediaMill concept detection approach is our top-performing bag-of-words system of last year, which uses multiple color descriptors, codebooks with soft-assignment, and kernel-based supervised learning. We improve upon this baseline system by exploring two novel research directions. Firstly, we study a multi-modal extension by the inclusion of 20 audio concepts and fusing using two novel multi-kernel supervised learning methods. Secondly, with the help of recently proposed algorithmic refinements of bag-of-words, a bag-of-words GPU implementation, and compute clusters, we scale-up the amount of visual information analyzed by an order of magnitude, to a total of 1,000,000 i-frames. Our experiments evaluate the merit of these new components, ultimately leading to 64 robust concept detectors for video retrieval. For retrieval, a robust but limited set of concept detectors necessitates the need to rely on as many auxiliary information channels as possible. For automatic search we therefore explore how we can learn to rank various information channels simultaneously to maximize video search results for a given topic. To improve the video retrieval results further, our interactive search experiments investigate the roles of visualizing preview results for a certain browse-dimension and relevance feedback mechanisms that learn to solve complex search topics by analysis from user browsing behavior. The 2009 edition of the TRECVID benchmark has again been a fruitful participation for the MediaMill team, resulting in the top ranking for both concept detection and interactive search.

2010

With the exponential growth of video data on the World Wide Web comes the challenge of efficient methods in video content management, content-based video search, filtering and browsing. But, video data often lacks sufficient metadata to open up the video content and to enable pinpoint content-based search. With the advent of the 'web of data' as an extension of the current WWW new data sources can be exploited by semantically interconnecting video metadata with the web of data. Thus, enabling better access to video repositories by deploying semantic search technologies and improving the user's search experience by supporting exploratory search strategies. We have developed the prototype semantic video search engine 'yovisto' that demonstrates the advantages of semantically enhanced exploratory video search and enables investigative navigation and browsing in large video repositories.

This paper gives a brief overview of various videos recommendation and Re-ranking techniques. It presents an advice framework which has been created to study examination addresses in the field of news feature suggestion and personalization. The framework is concentrated around semantically advanced feature information that allow look into on semantic models for flexible intelligent frameworks. It is frequently conceivable to enhance the recovery execution by re-positioning the examples. We proposed a re-positioning strategy that enhances the execution of semantic feature indexing and recovery by reassessing the scores of the shots by the homogeneity and the way of the feature they fit in with. Contradistinction with past works the proposed strategy gives a system to the re-positioning through the homogeneous circulation of feature shots content in a worldly arrangement. INTRODUCTION In web applications, request is submitted to web searchers to address the information needs of customers. Then again, on occasion inquiries may not unequivocally identify with customer's specific information needs since various obscure requests may cover a broad point and different customers may need to get information on differing perspectives when they submit the same request. For example, when the inquiry "the sun" is submitted to a web pursuit apparatus, a couple of customers need to discover the presentation page of an United Kingdom day by day paper, while a couple of others have to take in the trademark data of the sun. Video re-situating, as an issue methodology to upgrade the eventual outcomes of electronic video look for, has been grasped by force business web inquiry instruments. Given an inquiry definitive word pool of videos is at first recuperated by the web record concentrated around printed information. By asking the customer to pick a request video from the pool the remaining videos are resituated concentrated around their visual resemblances with the inquiry video. A critical test is that the comparable qualities of visual contrivances don't well relate with videos semantic ramifications which decode customers interest desire. In this project, we propose a novel videos re-situating, framework, which characteristically separated from the net learns unique visual semantic spaces for assorted inquiry definitive words through catch phrase augmentations. The visual characteristics of videos are expected into their related visual semantic spaces to get semantic imprints. At the online stage, videos are re-situated by taking a gender at their semantic imprints procured from the visual semantic space brought up by the inquiry urgent word. The new approach on a very basic level upgrades both the precision and capability of gimmick re-situating.

Journal of Imaging

This paper describes in detail VISIONE, a video search system that allows users to search for videos using textual keywords, the occurrence of objects and their spatial relationships, the occurrence of colors and their spatial relationships, and image similarity. These modalities can be combined together to express complex queries and meet users’ needs. The peculiarity of our approach is that we encode all information extracted from the keyframes, such as visual deep features, tags, color and object locations, using a convenient textual encoding that is indexed in a single text retrieval engine. This offers great flexibility when results corresponding to various parts of the query (visual, text and locations) need to be merged. In addition, we report an extensive analysis of the retrieval performance of the system, using the query logs generated during the Video Browser Showdown (VBS) 2019 competition. This allowed us to fine-tune the system by choosing the optimal parameters and st...