Artistic media play an important role in recognizing and classifying artworks in many artwork cla... more Artistic media play an important role in recognizing and classifying artworks in many artwork classification works and public artwork databases. We employ deep CNN structure to recognize artistic media from artworks and to classify them into predetermined categories. For this purpose, we define basic artistic media as oilpaint brush, pastel, pencil and watercolor and build artwork image dataset by collecting artwork images from various websites. To build our classifier, we implement various recent deep CNN structures and compare their performances. Among them, we select DenseNet, which shows best performance for recognizing artistic media. Through the human baseline experiment, we show that the performance of our classifier is compatible with that of trained human. Furthermore, we also show that our classifier shows a similar recognition and classification pattern with human in terms of well-classifying media, ill-classifying media, confusing pair and confusing case. We also collect synthesized oilpaint artwork images from fourteen important oilpaint literatures and apply them to our classifier. Our classifier shows a meaningful performance, which will lead to an evaluation scheme for the artistic media simulation techniques of non-photorealistic rendering (NPR) society.
We present a saliency-based patch sampling strategy for recognizing artistic media from artwork i... more We present a saliency-based patch sampling strategy for recognizing artistic media from artwork images using a deep media recognition model, which is composed of several deep convolutional neural network-based recognition modules. The decisions from the individual modules are merged into the final decision of the model. To sample a suitable patch for the input of the module, we devise a strategy that samples patches with high probabilities of containing distinctive media stroke patterns for artistic media without distortion, as media stroke patterns are key for media recognition. We design this strategy by collecting human-selected ground truth patches and analyzing the distribution of the saliency values of the patches. From this analysis, we build a strategy that samples patches that have a high probability of containing media stroke patterns. We prove that our strategy shows best performance among the existing patch sampling strategies and that our strategy shows a consistent rec...
We present a legorization framework that produces a LEGO model from user-specified 3D mesh model.... more We present a legorization framework that produces a LEGO model from user-specified 3D mesh model. Our framework is composed of two stages: voxelization and legorization. In the voxelization, input 3D mesh is converted to a voxel model. To preserve the shape of the 3D mesh, we devise a silhouette fitting process for the initial voxel model. For legorization, we propose three objectives: stability, aesthetics and efficiency. These objectives are expressed in a tiling equation, which builds a LEGO model using layer-by-layer approach. We legorize five models including characters and buildings to prove the excellence of our framework.
Proceedings of the Computer Graphics International Conference, 2017
We present a legorization framework that produces a LEGO model from user-specified 3D mesh model.... more We present a legorization framework that produces a LEGO model from user-specified 3D mesh model. Our framework is composed of two stages: voxelization and legorization. In the voxelization, input 3D mesh is converted to a voxel model. To preserve the shape of the 3D mesh, we devise a silhouette fitting process for the initial voxel model. For legorization, we propose three objectives: stability, aesthetics and efficiency. These objectives are expressed in a tiling equation, which builds a LEGO model using layer-by-layer approach. We legorize five models including characters and buildings to prove the excellence of our framework.
We present a framework that generates a 2D Lego-compatible puzzle layout of greater than thousand... more We present a framework that generates a 2D Lego-compatible puzzle layout of greater than thousands pieces of bricks using a reinforcement learning technique. Many existing 2D legorization strategies have limitations in producing a Lego layout, which is composed of more than thousands of pieces. We attack this problem by employing a reinforcement learning technique, which accelerates the progress of various game strategies. We represent the legorization process as a game tree search problem, where each leaf node of the tree corresponds to a Lego layout. The goal of legorization is to find an optimal Lego layout that achieves maximum reward. To efficiently find a leaf node for the maximum reward layout, we reduce the search space using a dueling deep Q-Network (DQN), which is a widely used reinforcement learning model. Our framework is composed of a learning stage and a legorization stage. In the learning stage, we design a dueling DQN model and train this model using three heuristics...
An illustrative sketch style expresses important shapes and regions of objects and scenes with sa... more An illustrative sketch style expresses important shapes and regions of objects and scenes with salient lines and dark tones, while abstracting less important shapes and regions as vacant spaces. We present a framework that produces illustrative sketch styles from various photographs. Our framework is designed using a generative adversarial network (GAN), which comprised four modules: a style extraction module, a generator module, a discriminator module and RCCL module. We devise two key ideas to effectively extract illustrative sketch styles from sample artworks and to apply them to input photographs. The first idea is using an attention map that extracts the required style features from important shapes and regions of sample illustrative sketch styles. This attention map is used in the generator module of our framework for the effective production of illustrative sketch styles. The second idea is using a relaxed cycle consistency loss that evaluates the quality of the produced illu...
The advancement and popularity of computer games make game scene analysis one of the most interes... more The advancement and popularity of computer games make game scene analysis one of the most interesting research topics in the computer vision society. Among the various computer vision techniques, we employ object detection algorithms for the analysis, since they can both recognize and localize objects in a scene. However, applying the existing object detection algorithms for analyzing game scenes does not guarantee a desired performance, since the algorithms are trained using datasets collected from the real world. In order to achieve a desired performance for analyzing game scenes, we built a dataset by collecting game scenes and retrained the object detection algorithms pre-trained with the datasets from the real world. We selected five object detection algorithms, namely YOLOv3, Faster R-CNN, SSD, FPN and EfficientDet, and eight games from various game genres including first-person shooting, role-playing, sports, and driving. PascalVOC and MS COCO were employed for the pre-traini...
Electroencephalogram (EEG) biosignals are widely used to measure human emotional reactions. The r... more Electroencephalogram (EEG) biosignals are widely used to measure human emotional reactions. The recent progress of deep learning-based classification models has improved the accuracy of emotion recognition in EEG signals. We apply a deep learning-based emotion recognition model from EEG biosignals to prove that illustrated surgical images reduce the negative emotional reactions that the photographic surgical images generate. The strong negative emotional reactions caused by surgical images, which show the internal structure of the human body (including blood, flesh, muscle, fatty tissue, and bone) act as an obstacle in explaining the images to patients or communicating with the images with non-professional people. We claim that the negative emotional reactions generated by illustrated surgical images are less severe than those caused by raw surgical images. To demonstrate the difference in emotional reaction, we produce several illustrated surgical images from photographs and measur...
Visual contents such as movies and animation evoke various human emotions. We examine an argument... more Visual contents such as movies and animation evoke various human emotions. We examine an argument that the emotion from the visual contents may vary according to the contrast control of the scenes contained in the contents. We sample three emotions including positive, neutral and negative to prove our argument. We also sample several scenes of these emotions from visual contents and control the contrast of the scenes. We manipulate the contrast of the scenes and measure the change of valence and arousal from human participants who watch the contents using a deep emotion recognition module based on electroencephalography (EEG) signals. As a result, we conclude that the enhancement of contrast induces the increase of valence, while the reduction of contrast induces the decrease. Meanwhile, the contrast control affects arousal on a very minute scale.
KSII Transactions on Internet and Information Systems, 2020
Recognizing food from photographs presents many applications for machine learning, computer visio... more Recognizing food from photographs presents many applications for machine learning, computer vision and dietetics, etc. Recent progress of deep learning techniques accelerates the recognition of food in a great scale. We build a hierarchical structure composed of deep CNN to recognize and classify food from photographs. We build a dataset for Korean food of 18 classes, which are further categorized in 4 major classes. Our hierarchical recognizer classifies foods into four major classes in the first step. Each food in the major classes is further classified into the exact class in the second step. We employ DenseNet structure for the baseline of our recognizer. The hierarchical structure provides higher accuracy and F1 score than those from the single-structured recognizer.
Visual stimuli from photographs and artworks raise corresponding emotional responses. It is a lon... more Visual stimuli from photographs and artworks raise corresponding emotional responses. It is a long process to prove whether the emotions that arise from photographs and artworks are different or not. We answer this question by employing electroencephalogram (EEG)-based biosignals and a deep convolutional neural network (CNN)-based emotion recognition model. We employ Russell’s emotion model, which matches emotion keywords such as happy, calm or sad to a coordinate system whose axes are valence and arousal, respectively. We collect photographs and artwork images that match the emotion keywords and build eighteen one-minute video clips for nine emotion keywords for photographs and artwork. We hired forty subjects and executed tests about the emotional responses from the video clips. From the t-test on the results, we concluded that the valence shows difference, while the arousal does not.
We present a multi-column structured framework for recognizing artistic media from artwork images... more We present a multi-column structured framework for recognizing artistic media from artwork images. We design the column of our framework using a deep neural network. Our key idea is to recognize the distinctive stroke texture of an artistic medium, which plays a key role in distinguishing artistic media. Since stroke texture is in a local scale, the whole image is not proper for recognizing the texture. Therefore, we devise two ideas for our framework: Sampling patches from an input image and employing a Gram matrix to extract the texture. The patches sampled from an input artwork image are processed in the columns of our framework to make local decisions on the patch, and the local decisions from the patches are merged to make a final decision for the input artwork image. Furthermore, we employ a Gram matrix, which is known to effectively capture texture information, to improve the accuracy of recognition. Our framework is trained and tested using two real artwork image datasets: W...
We present a multi-column CNN-based model for emotion recognition from EEG signals. Recently, a d... more We present a multi-column CNN-based model for emotion recognition from EEG signals. Recently, a deep neural network is widely employed for extracting features and recognizing emotions from various biosignals including EEG signals. A decision from a single CNN-based emotion recognizing module shows improved accuracy than the conventional handcrafted feature-based modules. To further improve the accuracy of the CNN-based modules, we devise a multi-column structured model, whose decision is produced by a weighted sum of the decisions from individual recognizing modules. We apply the model to EEG signals from DEAP dataset for comparison and demonstrate the improved accuracy of our model.
KSII Transactions on Internet and Information Systems, 2017
We present a computerized framework for producing color pastel painting from the visual informati... more We present a computerized framework for producing color pastel painting from the visual information extracted from a photograph. To express color pastel painting, we propose a multi-layered framework where each layer possesses pastel stroke patterns of different colors. The stroke patterns in the separate layers are merged by a rendering equation based on a participating media rendering scheme. To produce the stroke patterns in each layer, we review the physical properties of pastels and the mechanism of a convolution framework, which is the most widely used scheme to simulate stick-shaped media such as pencils. We devise the following computational models to extend the convolution framework to produce pastel strokes: a bold noise model, which mimics heavy and clustered deposition of pigment, and a thick convolution filter model, which produces various pastel stroke patterns. We also design a stochastic color coordination scheme to mimic pastel artists' color expression and to separate strokes in different layers. To demonstrate the soundness of approach, we conduct several experiments using the models and compare the results with existing works or real pastel paintings. We present the results for several pastel paintings to demonstrate the excellent performance of our framework.
International Journal of Multimedia and Ubiquitous Engineering, 2013
The computation of saliency from an image and a video is an interesting challenge in image proces... more The computation of saliency from an image and a video is an interesting challenge in image processing and computer vision. Context-aware saliency, which addresses the saliency based on the geometric structure of an image, is known as one of the most powerful schemes for computing saliency. An obstacle of the context-aware scheme is the heavy computation load. We reduce computational loads in a great scale by applying the dart throwing algorithm.
International Journal of Multimedia and Ubiquitous Engineering, 2014
We present an improved sketch-based image search framework through which users can search their t... more We present an improved sketch-based image search framework through which users can search their target images from the images in database. Our basic approach is to search the database by comparing the user-created sketch with the graph extracted from the images in the database and estimating the similarity. The images of high similarities are suggested as the candidates that match the target image. To improve the accuracy of the matching process, we substitute the graph-based representation of images with vectorized coherent lines, which are known as one of the most precise schemes in extracting and describing important features in an image. By the experiments on 820 images of 32 categories, we prove that our scheme shows higher matching accuracy than the existing schemes.
International Journal of Control and Automation, 2014
We present a noise control algorithm that produces temporally coherent artistic styles on movie f... more We present a noise control algorithm that produces temporally coherent artistic styles on movie files. Our algorithm is devised using a template to preserve the temporal coherence of the artistic effects produced on the frames composing a video. In the first step, we produce a set of triangular meshes embedding the contents of a frame. We, then, generate noise in the coordinate system defined by the templates. We finally apply a texture coverage scheme to preserve the coherence of the noise distribution inside the templates. For the noise outside the templates, we propose local recursive filters that interpolate the noise between frames.
Communications in Computer and Information Science, 2012
In this paper, we present a real-time video abstraction framework that responds to the affection ... more In this paper, we present a real-time video abstraction framework that responds to the affection of the contents. For abstraction, we extract edges and quantize the color of each frame of video in real-time in order to emphasize the important part of the frame. The key of our scheme is how to process each frame in real-time. This purpose is achieved by using CUDA environment, which is the GPU programming environment supported by nVidia, Inc. Using CUDA, our scheme can process more than eight frames per second in producing affectively abstracted images.
Communications in Computer and Information Science, 2012
We produce color pencil drawings from photographs using feature strokes to create the salient lin... more We produce color pencil drawings from photographs using feature strokes to create the salient linear elements of the picture, and hatching strokes to fill areas. Both types of strokes are generated using line integral convolution (LIC). We improve previous LIC schemes by using feature lines to determine the integration direction, and by introducing grayscale noise into each CMY color channel. Our scheme can produce visually pleasing color pencil drawings in various styles.
Artistic media play an important role in recognizing and classifying artworks in many artwork cla... more Artistic media play an important role in recognizing and classifying artworks in many artwork classification works and public artwork databases. We employ deep CNN structure to recognize artistic media from artworks and to classify them into predetermined categories. For this purpose, we define basic artistic media as oilpaint brush, pastel, pencil and watercolor and build artwork image dataset by collecting artwork images from various websites. To build our classifier, we implement various recent deep CNN structures and compare their performances. Among them, we select DenseNet, which shows best performance for recognizing artistic media. Through the human baseline experiment, we show that the performance of our classifier is compatible with that of trained human. Furthermore, we also show that our classifier shows a similar recognition and classification pattern with human in terms of well-classifying media, ill-classifying media, confusing pair and confusing case. We also collect synthesized oilpaint artwork images from fourteen important oilpaint literatures and apply them to our classifier. Our classifier shows a meaningful performance, which will lead to an evaluation scheme for the artistic media simulation techniques of non-photorealistic rendering (NPR) society.
We present a saliency-based patch sampling strategy for recognizing artistic media from artwork i... more We present a saliency-based patch sampling strategy for recognizing artistic media from artwork images using a deep media recognition model, which is composed of several deep convolutional neural network-based recognition modules. The decisions from the individual modules are merged into the final decision of the model. To sample a suitable patch for the input of the module, we devise a strategy that samples patches with high probabilities of containing distinctive media stroke patterns for artistic media without distortion, as media stroke patterns are key for media recognition. We design this strategy by collecting human-selected ground truth patches and analyzing the distribution of the saliency values of the patches. From this analysis, we build a strategy that samples patches that have a high probability of containing media stroke patterns. We prove that our strategy shows best performance among the existing patch sampling strategies and that our strategy shows a consistent rec...
We present a legorization framework that produces a LEGO model from user-specified 3D mesh model.... more We present a legorization framework that produces a LEGO model from user-specified 3D mesh model. Our framework is composed of two stages: voxelization and legorization. In the voxelization, input 3D mesh is converted to a voxel model. To preserve the shape of the 3D mesh, we devise a silhouette fitting process for the initial voxel model. For legorization, we propose three objectives: stability, aesthetics and efficiency. These objectives are expressed in a tiling equation, which builds a LEGO model using layer-by-layer approach. We legorize five models including characters and buildings to prove the excellence of our framework.
Proceedings of the Computer Graphics International Conference, 2017
We present a legorization framework that produces a LEGO model from user-specified 3D mesh model.... more We present a legorization framework that produces a LEGO model from user-specified 3D mesh model. Our framework is composed of two stages: voxelization and legorization. In the voxelization, input 3D mesh is converted to a voxel model. To preserve the shape of the 3D mesh, we devise a silhouette fitting process for the initial voxel model. For legorization, we propose three objectives: stability, aesthetics and efficiency. These objectives are expressed in a tiling equation, which builds a LEGO model using layer-by-layer approach. We legorize five models including characters and buildings to prove the excellence of our framework.
We present a framework that generates a 2D Lego-compatible puzzle layout of greater than thousand... more We present a framework that generates a 2D Lego-compatible puzzle layout of greater than thousands pieces of bricks using a reinforcement learning technique. Many existing 2D legorization strategies have limitations in producing a Lego layout, which is composed of more than thousands of pieces. We attack this problem by employing a reinforcement learning technique, which accelerates the progress of various game strategies. We represent the legorization process as a game tree search problem, where each leaf node of the tree corresponds to a Lego layout. The goal of legorization is to find an optimal Lego layout that achieves maximum reward. To efficiently find a leaf node for the maximum reward layout, we reduce the search space using a dueling deep Q-Network (DQN), which is a widely used reinforcement learning model. Our framework is composed of a learning stage and a legorization stage. In the learning stage, we design a dueling DQN model and train this model using three heuristics...
An illustrative sketch style expresses important shapes and regions of objects and scenes with sa... more An illustrative sketch style expresses important shapes and regions of objects and scenes with salient lines and dark tones, while abstracting less important shapes and regions as vacant spaces. We present a framework that produces illustrative sketch styles from various photographs. Our framework is designed using a generative adversarial network (GAN), which comprised four modules: a style extraction module, a generator module, a discriminator module and RCCL module. We devise two key ideas to effectively extract illustrative sketch styles from sample artworks and to apply them to input photographs. The first idea is using an attention map that extracts the required style features from important shapes and regions of sample illustrative sketch styles. This attention map is used in the generator module of our framework for the effective production of illustrative sketch styles. The second idea is using a relaxed cycle consistency loss that evaluates the quality of the produced illu...
The advancement and popularity of computer games make game scene analysis one of the most interes... more The advancement and popularity of computer games make game scene analysis one of the most interesting research topics in the computer vision society. Among the various computer vision techniques, we employ object detection algorithms for the analysis, since they can both recognize and localize objects in a scene. However, applying the existing object detection algorithms for analyzing game scenes does not guarantee a desired performance, since the algorithms are trained using datasets collected from the real world. In order to achieve a desired performance for analyzing game scenes, we built a dataset by collecting game scenes and retrained the object detection algorithms pre-trained with the datasets from the real world. We selected five object detection algorithms, namely YOLOv3, Faster R-CNN, SSD, FPN and EfficientDet, and eight games from various game genres including first-person shooting, role-playing, sports, and driving. PascalVOC and MS COCO were employed for the pre-traini...
Electroencephalogram (EEG) biosignals are widely used to measure human emotional reactions. The r... more Electroencephalogram (EEG) biosignals are widely used to measure human emotional reactions. The recent progress of deep learning-based classification models has improved the accuracy of emotion recognition in EEG signals. We apply a deep learning-based emotion recognition model from EEG biosignals to prove that illustrated surgical images reduce the negative emotional reactions that the photographic surgical images generate. The strong negative emotional reactions caused by surgical images, which show the internal structure of the human body (including blood, flesh, muscle, fatty tissue, and bone) act as an obstacle in explaining the images to patients or communicating with the images with non-professional people. We claim that the negative emotional reactions generated by illustrated surgical images are less severe than those caused by raw surgical images. To demonstrate the difference in emotional reaction, we produce several illustrated surgical images from photographs and measur...
Visual contents such as movies and animation evoke various human emotions. We examine an argument... more Visual contents such as movies and animation evoke various human emotions. We examine an argument that the emotion from the visual contents may vary according to the contrast control of the scenes contained in the contents. We sample three emotions including positive, neutral and negative to prove our argument. We also sample several scenes of these emotions from visual contents and control the contrast of the scenes. We manipulate the contrast of the scenes and measure the change of valence and arousal from human participants who watch the contents using a deep emotion recognition module based on electroencephalography (EEG) signals. As a result, we conclude that the enhancement of contrast induces the increase of valence, while the reduction of contrast induces the decrease. Meanwhile, the contrast control affects arousal on a very minute scale.
KSII Transactions on Internet and Information Systems, 2020
Recognizing food from photographs presents many applications for machine learning, computer visio... more Recognizing food from photographs presents many applications for machine learning, computer vision and dietetics, etc. Recent progress of deep learning techniques accelerates the recognition of food in a great scale. We build a hierarchical structure composed of deep CNN to recognize and classify food from photographs. We build a dataset for Korean food of 18 classes, which are further categorized in 4 major classes. Our hierarchical recognizer classifies foods into four major classes in the first step. Each food in the major classes is further classified into the exact class in the second step. We employ DenseNet structure for the baseline of our recognizer. The hierarchical structure provides higher accuracy and F1 score than those from the single-structured recognizer.
Visual stimuli from photographs and artworks raise corresponding emotional responses. It is a lon... more Visual stimuli from photographs and artworks raise corresponding emotional responses. It is a long process to prove whether the emotions that arise from photographs and artworks are different or not. We answer this question by employing electroencephalogram (EEG)-based biosignals and a deep convolutional neural network (CNN)-based emotion recognition model. We employ Russell’s emotion model, which matches emotion keywords such as happy, calm or sad to a coordinate system whose axes are valence and arousal, respectively. We collect photographs and artwork images that match the emotion keywords and build eighteen one-minute video clips for nine emotion keywords for photographs and artwork. We hired forty subjects and executed tests about the emotional responses from the video clips. From the t-test on the results, we concluded that the valence shows difference, while the arousal does not.
We present a multi-column structured framework for recognizing artistic media from artwork images... more We present a multi-column structured framework for recognizing artistic media from artwork images. We design the column of our framework using a deep neural network. Our key idea is to recognize the distinctive stroke texture of an artistic medium, which plays a key role in distinguishing artistic media. Since stroke texture is in a local scale, the whole image is not proper for recognizing the texture. Therefore, we devise two ideas for our framework: Sampling patches from an input image and employing a Gram matrix to extract the texture. The patches sampled from an input artwork image are processed in the columns of our framework to make local decisions on the patch, and the local decisions from the patches are merged to make a final decision for the input artwork image. Furthermore, we employ a Gram matrix, which is known to effectively capture texture information, to improve the accuracy of recognition. Our framework is trained and tested using two real artwork image datasets: W...
We present a multi-column CNN-based model for emotion recognition from EEG signals. Recently, a d... more We present a multi-column CNN-based model for emotion recognition from EEG signals. Recently, a deep neural network is widely employed for extracting features and recognizing emotions from various biosignals including EEG signals. A decision from a single CNN-based emotion recognizing module shows improved accuracy than the conventional handcrafted feature-based modules. To further improve the accuracy of the CNN-based modules, we devise a multi-column structured model, whose decision is produced by a weighted sum of the decisions from individual recognizing modules. We apply the model to EEG signals from DEAP dataset for comparison and demonstrate the improved accuracy of our model.
KSII Transactions on Internet and Information Systems, 2017
We present a computerized framework for producing color pastel painting from the visual informati... more We present a computerized framework for producing color pastel painting from the visual information extracted from a photograph. To express color pastel painting, we propose a multi-layered framework where each layer possesses pastel stroke patterns of different colors. The stroke patterns in the separate layers are merged by a rendering equation based on a participating media rendering scheme. To produce the stroke patterns in each layer, we review the physical properties of pastels and the mechanism of a convolution framework, which is the most widely used scheme to simulate stick-shaped media such as pencils. We devise the following computational models to extend the convolution framework to produce pastel strokes: a bold noise model, which mimics heavy and clustered deposition of pigment, and a thick convolution filter model, which produces various pastel stroke patterns. We also design a stochastic color coordination scheme to mimic pastel artists' color expression and to separate strokes in different layers. To demonstrate the soundness of approach, we conduct several experiments using the models and compare the results with existing works or real pastel paintings. We present the results for several pastel paintings to demonstrate the excellent performance of our framework.
International Journal of Multimedia and Ubiquitous Engineering, 2013
The computation of saliency from an image and a video is an interesting challenge in image proces... more The computation of saliency from an image and a video is an interesting challenge in image processing and computer vision. Context-aware saliency, which addresses the saliency based on the geometric structure of an image, is known as one of the most powerful schemes for computing saliency. An obstacle of the context-aware scheme is the heavy computation load. We reduce computational loads in a great scale by applying the dart throwing algorithm.
International Journal of Multimedia and Ubiquitous Engineering, 2014
We present an improved sketch-based image search framework through which users can search their t... more We present an improved sketch-based image search framework through which users can search their target images from the images in database. Our basic approach is to search the database by comparing the user-created sketch with the graph extracted from the images in the database and estimating the similarity. The images of high similarities are suggested as the candidates that match the target image. To improve the accuracy of the matching process, we substitute the graph-based representation of images with vectorized coherent lines, which are known as one of the most precise schemes in extracting and describing important features in an image. By the experiments on 820 images of 32 categories, we prove that our scheme shows higher matching accuracy than the existing schemes.
International Journal of Control and Automation, 2014
We present a noise control algorithm that produces temporally coherent artistic styles on movie f... more We present a noise control algorithm that produces temporally coherent artistic styles on movie files. Our algorithm is devised using a template to preserve the temporal coherence of the artistic effects produced on the frames composing a video. In the first step, we produce a set of triangular meshes embedding the contents of a frame. We, then, generate noise in the coordinate system defined by the templates. We finally apply a texture coverage scheme to preserve the coherence of the noise distribution inside the templates. For the noise outside the templates, we propose local recursive filters that interpolate the noise between frames.
Communications in Computer and Information Science, 2012
In this paper, we present a real-time video abstraction framework that responds to the affection ... more In this paper, we present a real-time video abstraction framework that responds to the affection of the contents. For abstraction, we extract edges and quantize the color of each frame of video in real-time in order to emphasize the important part of the frame. The key of our scheme is how to process each frame in real-time. This purpose is achieved by using CUDA environment, which is the GPU programming environment supported by nVidia, Inc. Using CUDA, our scheme can process more than eight frames per second in producing affectively abstracted images.
Communications in Computer and Information Science, 2012
We produce color pencil drawings from photographs using feature strokes to create the salient lin... more We produce color pencil drawings from photographs using feature strokes to create the salient linear elements of the picture, and hatching strokes to fill areas. Both types of strokes are generated using line integral convolution (LIC). We improve previous LIC schemes by using feature lines to determine the integration direction, and by introducing grayscale noise into each CMY color channel. Our scheme can produce visually pleasing color pencil drawings in various styles.
Uploads
Papers by Heekyung Yang