Abstract. Various efficient game problem solvers are based on PN-Search. Especially depth-first v... more Abstract. Various efficient game problem solvers are based on PN-Search. Especially depth-first versions of PN-Search like DF-PN or PDS – contrary to other known techniques – are able to solve really hard problems. However, the performance of DF-PN and PDS decreases dramatically when the search space significantly exceeds available memory. A simple trick to overcome this problem is presented. Experiments on Atari Go and Lines of Action show great practical value of the proposed enhancement. 1
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021
Quantization has become a popular technique to compress neural networks and reduce compute cost, ... more Quantization has become a popular technique to compress neural networks and reduce compute cost, but most prior work focuses on studying quantization without changing the network size. Many real-world applications of neural networks have compute cost and memory budgets, which can be traded off with model quality by changing the number of parameters. In this work, we use ResNet as a case study to systematically investigate the effects of quantization on inference compute cost-quality tradeoff curves. Our results suggest that for each bfloat16 ResNet model, there are quantized models with lower cost and higher accuracy; in other words, the bfloat16 compute cost-quality tradeoff curve is Pareto-dominated by the 4-bit and 8-bit curves, with models primarily quantized to 4-bit yielding the best Pareto curve. Furthermore, we achieve stateof-the-art results on ImageNet for 4-bit ResNet-50 with quantization-aware training, obtaining a top-1 eval accuracy of 77.09%. We demonstrate the regularizing effect of quantization by measuring the generalization gap. The quantization method we used is optimized for practicality: It requires little tuning and is designed with hardware capabilities in mind. Our work motivates further research into optimal numeric formats for quantization, as well as the development of machine learning accelerators supporting these formats. As part of this work, we contribute a quantization library written in JAX, which is open-sourced at https : / / github. com / google-research / google-research/tree/master/aqt.
Research on computer Go is almost as old as research on the other board games. But until recent d... more Research on computer Go is almost as old as research on the other board games. But until recent developments of Monte Carlo Go, programs were not considered to be particularly strong. The key to the current programs strength is the quality of the moves in the game continuation samples. In the state of the art programs, moves in samples are chosen based on a local (to the move) Go heuristics. This heuristics are usually learnt from game records of good players using Minorization-Maximization (MM) algorithm. MM is the state of the art but it can process only a limited number of games due to memory constraints of existing hardware. In this thesis, we design and present a better, online algorithm, capable of learning from unconstrained data set. We prove its convergence. State of the art Go programs play on a regular, 19 × 19 Go board on the level of a strong club player and on a small 9 × 9 board on a near professional level. Such discrepancy in playing strength is uncommon among human players, who tend to play on a similar level regardless of the board size. We attribute Monte Carlo Go weakness on a regular board to the ineffectiveness of a global Monte Carlo Tree Search (MCTS). 19 × 19 Go positions usually consist of a multiple smaller independent subgames that human players can analyze independently. MCTS is capable of analyzing any of them separately but globally it is ineffective due to combinatorial explosion when subgames' moves can interlace. We review two decomposition-based, promising algorithms trying to work around that ineffectiveness by exploiting locality in Go positions. For each of them, we discover and describe the reason why it can't work well. Finally we propose a new, divide and conquer algorithm based on Combinatorial Game Theory. We show that it must converge to a near-optimal strategy, and experimentally show the convergence on a toy example.
Various efficient game problem solvers are based on PN-Search. Especially depth-first versions of... more Various efficient game problem solvers are based on PN-Search. Especially depth-first versions of PN-Search like DF-PN or PDS -contrary to other known techniques -are able to solve really hard problems. However, the performance of DF-PN and PDS decreases dramatically when the search space significantly exceeds available memory. A simple trick to overcome this problem is presented. Experiments on Atari Go and Lines of Action show great practical value of the proposed enhancement.
Monte-Carlo Tree Search is a very successful game playing algorithm. Unfortunately it suffers fro... more Monte-Carlo Tree Search is a very successful game playing algorithm. Unfortunately it suffers from the horizon effect: some important tactical sequences may be delayed beyond the depth of the search tree, causing evaluation errors. Temporal-difference search with a function approximation is a method that was proposed to overcome these weaknesses, by adaptively changing the simulation policy outside the tree. In this paper we present an experimental evidence demonstrating that a temporal difference search may fail to find an optimal policy, even in very simple game positions. Classical temporal difference algorithms try to evaluate a local situation with a numerical value, but, as it appears, a single number is not enough to model the dynamics of a partial two-player game state. As a solution we propose to replace numerical values by approximate thermographs. With this richer representation of partial states, reinforcement-learning algorithms converge and accurately represent dynamics of states, allowing to find an optimal policy.
Top-1 ImageNet optimization promotes enormous networks that may be impractical in inference setti... more Top-1 ImageNet optimization promotes enormous networks that may be impractical in inference settings. Binary neural networks (BNNs) have the potential to significantly lower the compute intensity but existing models suffer from low quality. To overcome this deficiency, we propose PokeConv, a binary convolution block which improves quality of BNNs by techniques such as adding multiple residual paths, and tuning the activation function. We apply it to ResNet-50 and optimize ResNet’s initial convolutional layer which is hard to binarize. We name the resulting network family PokeBNN1. These techniques are chosen to yield favorable improvements in both top-1 accuracy and the network’s cost. In order to enable joint optimization of the cost together with accuracy, we define arithmetic computation effort (ACE), a hardwareand energy-inspired cost metric for quantized and binarized networks. We also identify a need to optimize an under-explored hyper-parameter controlling the binarization gr...
Abstract. Various efficient game problem solvers are based on PN-Search. Especially depth-first v... more Abstract. Various efficient game problem solvers are based on PN-Search. Especially depth-first versions of PN-Search like DF-PN or PDS – contrary to other known techniques – are able to solve really hard problems. However, the performance of DF-PN and PDS decreases dramatically when the search space significantly exceeds available memory. A simple trick to overcome this problem is presented. Experiments on Atari Go and Lines of Action show great practical value of the proposed enhancement. 1
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021
Quantization has become a popular technique to compress neural networks and reduce compute cost, ... more Quantization has become a popular technique to compress neural networks and reduce compute cost, but most prior work focuses on studying quantization without changing the network size. Many real-world applications of neural networks have compute cost and memory budgets, which can be traded off with model quality by changing the number of parameters. In this work, we use ResNet as a case study to systematically investigate the effects of quantization on inference compute cost-quality tradeoff curves. Our results suggest that for each bfloat16 ResNet model, there are quantized models with lower cost and higher accuracy; in other words, the bfloat16 compute cost-quality tradeoff curve is Pareto-dominated by the 4-bit and 8-bit curves, with models primarily quantized to 4-bit yielding the best Pareto curve. Furthermore, we achieve stateof-the-art results on ImageNet for 4-bit ResNet-50 with quantization-aware training, obtaining a top-1 eval accuracy of 77.09%. We demonstrate the regularizing effect of quantization by measuring the generalization gap. The quantization method we used is optimized for practicality: It requires little tuning and is designed with hardware capabilities in mind. Our work motivates further research into optimal numeric formats for quantization, as well as the development of machine learning accelerators supporting these formats. As part of this work, we contribute a quantization library written in JAX, which is open-sourced at https : / / github. com / google-research / google-research/tree/master/aqt.
Research on computer Go is almost as old as research on the other board games. But until recent d... more Research on computer Go is almost as old as research on the other board games. But until recent developments of Monte Carlo Go, programs were not considered to be particularly strong. The key to the current programs strength is the quality of the moves in the game continuation samples. In the state of the art programs, moves in samples are chosen based on a local (to the move) Go heuristics. This heuristics are usually learnt from game records of good players using Minorization-Maximization (MM) algorithm. MM is the state of the art but it can process only a limited number of games due to memory constraints of existing hardware. In this thesis, we design and present a better, online algorithm, capable of learning from unconstrained data set. We prove its convergence. State of the art Go programs play on a regular, 19 × 19 Go board on the level of a strong club player and on a small 9 × 9 board on a near professional level. Such discrepancy in playing strength is uncommon among human players, who tend to play on a similar level regardless of the board size. We attribute Monte Carlo Go weakness on a regular board to the ineffectiveness of a global Monte Carlo Tree Search (MCTS). 19 × 19 Go positions usually consist of a multiple smaller independent subgames that human players can analyze independently. MCTS is capable of analyzing any of them separately but globally it is ineffective due to combinatorial explosion when subgames' moves can interlace. We review two decomposition-based, promising algorithms trying to work around that ineffectiveness by exploiting locality in Go positions. For each of them, we discover and describe the reason why it can't work well. Finally we propose a new, divide and conquer algorithm based on Combinatorial Game Theory. We show that it must converge to a near-optimal strategy, and experimentally show the convergence on a toy example.
Various efficient game problem solvers are based on PN-Search. Especially depth-first versions of... more Various efficient game problem solvers are based on PN-Search. Especially depth-first versions of PN-Search like DF-PN or PDS -contrary to other known techniques -are able to solve really hard problems. However, the performance of DF-PN and PDS decreases dramatically when the search space significantly exceeds available memory. A simple trick to overcome this problem is presented. Experiments on Atari Go and Lines of Action show great practical value of the proposed enhancement.
Monte-Carlo Tree Search is a very successful game playing algorithm. Unfortunately it suffers fro... more Monte-Carlo Tree Search is a very successful game playing algorithm. Unfortunately it suffers from the horizon effect: some important tactical sequences may be delayed beyond the depth of the search tree, causing evaluation errors. Temporal-difference search with a function approximation is a method that was proposed to overcome these weaknesses, by adaptively changing the simulation policy outside the tree. In this paper we present an experimental evidence demonstrating that a temporal difference search may fail to find an optimal policy, even in very simple game positions. Classical temporal difference algorithms try to evaluate a local situation with a numerical value, but, as it appears, a single number is not enough to model the dynamics of a partial two-player game state. As a solution we propose to replace numerical values by approximate thermographs. With this richer representation of partial states, reinforcement-learning algorithms converge and accurately represent dynamics of states, allowing to find an optimal policy.
Top-1 ImageNet optimization promotes enormous networks that may be impractical in inference setti... more Top-1 ImageNet optimization promotes enormous networks that may be impractical in inference settings. Binary neural networks (BNNs) have the potential to significantly lower the compute intensity but existing models suffer from low quality. To overcome this deficiency, we propose PokeConv, a binary convolution block which improves quality of BNNs by techniques such as adding multiple residual paths, and tuning the activation function. We apply it to ResNet-50 and optimize ResNet’s initial convolutional layer which is hard to binarize. We name the resulting network family PokeBNN1. These techniques are chosen to yield favorable improvements in both top-1 accuracy and the network’s cost. In order to enable joint optimization of the cost together with accuracy, we define arithmetic computation effort (ACE), a hardwareand energy-inspired cost metric for quantized and binarized networks. We also identify a need to optimize an under-explored hyper-parameter controlling the binarization gr...
Uploads
Papers by Łukasz Lew