Artistic ideation based on computer vision methods

In this paper we explore supervised learning techniques that are able to classify fine-art paintings by the general subject matter of the painting (e.g., landscape, people, seascape and still life). Classifying art paintings by semantic category may pose unique challenges because art is subjective and highly interpretive. State-of-the-art feature extraction and encoding techniques used for object and scene recognition in photographic images are evaluated for their potential use for classifying art paintings. In this work we evaluate several types of features individually and also in combinations to reveal the benefit of complimentary information. Feature ranking techniques are implemented as a means for identifying the most important features between any two labels in the data set. In order to compute a final ranking of the most relevant features for all class labels, a metric is proposed using the principal components of the combined feature vector to prioritize the final ranking of the top 'k' features across all class label pairs. Classification algorithms used for evaluation include a soft margin linear SVM and L-2 regularized logistic regression. Experimental results show that several feature classes can be successfully used to classify art paintings with improved accuracy when multiple features are combined, ranked and prioritized to form a final feature vector.

Lecture Notes in Computer Science, 2012

Artistic image understanding is an interdisciplinary research field of increasing importance for the computer vision and the art history communities. For computer vision scientists, this problem offers challenges where new techniques can be developed; and for the art history community new automatic art analysis tools can be developed. On the positive side, artistic images are generally constrained by compositional rules and artistic themes. However, the low-level texture and color features exploited for photographic image analysis are not as effective because of inconsistent color and texture patterns describing the visual classes in artistic images. In this work, we present a new database of monochromatic artistic images containing 988 images with a global semantic annotation, a local compositional annotation, and a pose annotation of human subjects and animal types. In total, 75 visual classes are annotated, from which 27 are related to the theme of the art image, and 48 are visual classes that can be localized in the image with bounding boxes. Out of these 48 classes, 40 have pose annotation, with 37 denoting human subjects and 3 representing animal types. We also provide a complete evaluation of several algorithms recently proposed for image annotation and retrieval. We then present an algorithm achieving remarkable performance over the most successful algorithm hitherto proposed for this problem. Our main goal with this paper is to make this database, the evaluation process, and the benchmark results available for the computer vision community.

Leonardo, 2017

This study uses computer vision models, which to some extent simulate the initial stages of human visual perception, to help categorize data in large sets of images of artworks by the artist Antoni Tàpies. The images have been analyzed on the basis of their compositional, chromatic and organizational characteristics, without textual notes, so that the analogies found may take us closer to, and help us to understand, the creator’s original values. The system as programmed can assist the specialist by establishing analogies between different artists or periods using the same criteria.

Pattern Recognition Letters, 2014

Ancient paintings are valuable for historians and archeologists to study the humanities, customs and economy of the corresponding eras. For this purpose, it is important to first determine the era in which a painting was drawn. This problem can be very challenging when the paintings from different eras present a same topic and only show subtle difference in terms of the painting styles. In this paper, we propose a novel computational approach to address this problem by using the appearance and shape features extracted from the paintings. In this approach, we first extract the appearance and shape features using the SIFT and kAS descriptors, respectively. We then encode these features with deep learning in an unsupervised way. Finally, we combine all the features in the form of bag-of-visual-words and train a classifier in a supervised fashion. In the experiments, we collect 660 Flying-Apsaras paintings from Mogao Grottoes in Dunhuang, China and classify them into three different eras, with very promising results.

Proceedings of the 21st International Conference on Pattern Recognition, 2012

This thesis presents a comparative study of different classification methodologies for the task of fine-art genre classification. The problem of painting classification involves classifying new unknown paintings among different art genres. Two-level comparative study is performed for this classification problem. The first level reviews the performance of discriminative vs. generative models while the second level touches the features aspect of the paintings and compares Semantic-level features vs low-level and intermediate-level features present in the painting. Three models are studied and compared, namely-1) A Discriminative model using a Bag-of-Words (BoW) approach; 2) A Generative model using BoW; 3) Discriminative model using Semantic-level features. Various experiments and techniques like Bag of Words model, Topic models and Classeme features are employed to get insights into potential of these automatic classification techniques for painting styles.

The categorization of art (paintings, literature) into distinct styles such as Expressionism, or Surrealism has had a profound influence on how art is presented, marketed, analyzed, and historicized. Here, we present results from human and computational experiments with the goal of determining to which degree such categories can be explained by simple, low-level appearance information in the image. Following experimental methods from perceptual psychology on category formation, naive, non-expert participants were first asked to sort printouts of artworks from different art periods into categories. Converting these data into similarity data and running a multi-dimensional scaling (MDS) analysis, we found distinct categories which corresponded sometimes surprisingly well to canonical art periods. The result was cross-validated on two complementary sets of artworks for two different groups of participants showing the stability of art interpretation. The second focus of this paper was on determining how far computational algorithms would be able to capture human performance or would be able in general to separate different art categories. Using several state-of-the-art algorithms from computer vision, we found that whereas low-level appearance information can give some clues about category membership, human grouping strategies included also much higher-level concepts.

Computers & …, 2009

The user has requested enhancement of the downloaded file. This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues.

Since the development of the first text-based image search on the internet, the area of image retrieval has come a long way to sophisticated content based image retrieval systems. On the other hand, the semantic gap causes that it is still not possible to create a system which can correctly identify any object in the image. However, this paper proposes a solution for classifying the one sort of objects -paintings. This approach includes segmentation of the painting from the image, creation of the descriptor file from the segmented painting, and classification of the painting by matching its descriptor file to the created database of descriptor files of original paintings. The segmentation of the painting is achieved with 3 preprocessing steps, followed by adjusted Hough transformation. For the estimation of key points and creation of the descriptor file, the SIFT (Scalable Invariant Feature Transform) or the SURF(Speeded Up Robust Features) technique is used.

Lecture Notes in Computer Science, 2014

We have approached the difficulties of automatic cataloguing of images on which the conception and design of sculptor M. Planas artistic production are based. In order to build up a visual vocabulary for basing image description on, we followed a procedure similar to the method Bag-of-Words (BOW). We have implemented a probabilistic latent semantic analysis (PLSA) that detects underlying topics in images. Whole image collection was clustered into different types that describe aesthetic preferences of the artist. The outcomes are promising, the described cataloguing method may provide new viewpoints for the artist in future works.

We present a novel method for generic visual categorization: the problem of identifying the object content of natural images while generalizing across variations inherent to the object class. This bag of keypoints method is based on vector quantization of affine invariant descriptors of image patches. We propose and compare two alternative implementations using different classifiers: Naïve Bayes and SVM. The main advantages of the method are that it is simple, computationally efficient and intrinsically invariant. We present results for simultaneously classifying seven semantic visual categories. These results clearly demonstrate that the method is robust to background clutter and produces good categorization accuracy even without exploiting geometric information.