Entropy and the Value of Information for Investors

Entropy, 2022

Information theory is a well-established method for the study of many phenomena and more than 70 years after Claude Shannon first described it in A Mathematical Theory of Communication it has been extended well beyond Shannon’s initial vision. It is now an interdisciplinary tool that is used from ‘causal’ information flow to inferring complex computational processes and it is common to see it play an important role in fields as diverse as neuroscience, artificial intelligence, quantum mechanics, and astrophysics. In this article, I provide a selective review of a specific aspect of information theory that has received less attention than many of the others: as a tool for understanding, modelling, and detecting non-linear phenomena in finance and economics. Although some progress has been made in this area, it is still an under-developed area that I argue has considerable scope for further development.

cs.cmu.edu

Investing to optimally maximize the growth rate of wealth based on sequences of event outcomes has many information-theoretic interpretations. Namely, the mutual information characterizes the benefit of additional side information being available when making investment decisions [1] in settings where the probabilistic relationships between side information and event outcomes are known. Additionally, the relative variant of the principle of maximum entropy [2] provides the optimal investment allocation in the more general setting where the relationships between side information and event outcomes are only partially known . In this paper, we build upon recent work characterizing the growth rates of investment in settings with inter-dependent side information and event outcome sequences . We consider the extension to settings with inter-dependent event outcomes and side information where the probabilistic relationships between side information and event outcomes are only partially known. We introduce the principle of minimum relative causal entropy to obtain the optimal worst-case investment allocations for this setting. We present efficient algorithms for obtaining these investment allocations using convex optimization techniques and dynamic programming that illustrates a close connection to optimal control theory.

We investigate entropy as a novel risk measure which explains the equity premium of securities and portfolios in a simpler way and at the same time with higher explanatory power than the beta parameter of the capital asset pricing. To measure the risk of an investment opportunity the portfolio theory applies the variance of the return, and show that the risk can be reduced by diversification and the systematic risk (beta) is applied as the risk measure. Entropy represents a measure of the uncertainty of a probability variable. Analogously, for asset pricing we define the continuous entropy as an alternative measure of risk. Our results show that the entropy is decreasing in the function of the number of securities involved into a portfolio similarly to the variance. In this empirical study we use the daily returns of 150 randomly selected securities for a period of 27 years. Our regression results show that the entropy has a higher explanatory power for the expected return than the CAPM beta.

Journal of Economic Dynamics and Control, 1997

We generalize the economic decision problem considered by in which a decision-maker chooses an action after observing a signal correlated to the state of nature. Unlike Blackwell's case where the feasible set is fixed, in our framework the feasible set of actions depends on the signal and the information system. We argue that such a framework has more significance to economic models. As was demonstrated by Hirshleifer (1971) in such cases, contrary to Blackwell's well-known result. more information may be disadvantageous.

SSRN Electronic Journal, 1999

This paper studies decision problems under uncertainty where a decision-maker observes an imperfect signal about the true state of the world. We analyze the information preferences and information demand of such decision-makers, based on properties of their payoff functions. We restrict attention to "monotone decision problems," whereby the posterior beliefs induced by the signal can be ordered so that higher actions are chosen in response to higher signal realizations. Monotone decision problems are frequently encountered in economic modeling. We provide necessary and sufficient conditions for all decision makers with different classes of payoff functions to prefer one information structure to another. We also provide conditions under which two decision-makers in a given class can be ranked in terms of their marginal value for information and hence information demand. Applications and examples are given.

Journal of Financial and Quantitative Analysis, 2010

We study the consumption-investment problem of an agent with a constant relative risk aversion preference function, who possesses noisy information about the future prospects of a stock. We also solve for the value of information to the agent in closed form. We find that information can significantly alter consumption and asset allocation decisions. For reasonable parameter ranges, information increases consumption in the vicinity of 25%. Information can shift the portfolio weight on a stock from 0% to around 70%. Thus, depending on the stock beta, the weight on the market portfolio can be considerably reduced with information, causing the appearance of underdiversification. The model indicates that stock holdings of informed agents are positively related to wealth, unrelated to systematic risk, and negatively related to idiosyncratic uncertainty. We also show that the dollar value of information to the agent depends linearly on his wealth and decreases with both the propensity to i...

Physica A: Statistical Mechanics and its …, 2010

We present an expression of the economic concept of asymmetric information with which it is possible to derive the dynamical laws of an economy. To illustrate the utility of this approach we show how the assumption of optimal information flow leads to a general class of investment strategies including the well-known Q theory of Tobin. Novel consequences of this formalism include a natural definition of market efficiency and an uncertainty principle relating capital stock and investment flow.

Central European Journal of Economic Modelling and Econometrics, 2015

Various approaches have been introduced over the years to evaluate information in the expected utility framework. This paper analyzes the relationship between the degree of risk aversion and the selling price of information in a lottery setting with two actions. We show that the initial decision on the lottery as well as the attitude of the decision maker towards risk as a function of the initial wealth level are critical to characterizing this relationship. When the initial decision is to reject, a non-decreasingly risk averse decision maker asks for a higher selling price as he gets less risk averse. Conversely, when the initial decision is to accept, non-increasingly risk averse decision makers ask a higher selling price as they get more risk averse if information is collected on bounded lotteries. We also show that the assumption of the lower bound for lotteries can be relaxed for the quadratic utility family.

Economic Theory, 2001

In a game with rational expectations, individuals simultaneously refine their information with the information revealed by the strategies of other individuals. At a Nash equilibrium of a game with rational expectations, the information of individuals is essentially symmetric: the same profile of strategies is also an equilibrium of a game with symmetric information; and strategies are common knowledge. If each player has a veto act, which yields a minimum payoff that no other profile of strategies attains, then the veto profile is the only Nash equilibrium, and it is is an equilibrium with rational expectations and essentially symmetric information; which accounts for the impossibility of speculation.

2016

Dedicated to Professor Andrzej Lasota on his 70th birthday Expected utility maximization problems in mathematical finance lead to a generalization of the classical definition of entropy. It is demonstrated that a necessary and sufficient condition for the second law of thermodynamics to operate is that any one of the generalized entropies should tend to its minimum value of zero.

Lecture Notes in Computer Science, 2011

Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here we propose an axiomatic framework for bounded rational decision-making based on a thermodynamic interpretation of resource costs as information costs. We show that this axiomatic framework enforces a unique conversion law between utility and information, which can be characterized by a variational "free utility" principle akin to thermodynamical free energy. This variational principle constitutes a normative criterion that trades off utility and information costs, the latter measured by the Kullback-Leibler deviation between a distribution representing a desired policy and a reference distribution representing an initial default policy. We show that bounded optimal control solutions can be derived from this variational principle, which leads in general to stochastic policies. Furthermore, we show that risk-sensitive and robust (minimax) control schemes fall out naturally from this framework if the environment is considered as an adversarial opponent. When resource costs are ignored, the maximum expected utility principle is recovered. ⋆ A shortened version of this paper has been published in Lecture Notes on Artificial Intelligence 6830, pp. 269-274.

Journal of Economics, 2011

A decision maker faces two correlated risks and can obtain information on only one of them. Intuition suggests that the existence of a high correlation (in absolute value) between the risks should increase total information value. Indeed in such a case information about one risk induces a relevant information on the other one. Using a simple example, we show that this intuition is often correct, but that it can also be mitigated by other factors.

Entropy

In this paper we investigate the relationship between the information entropy of the distribution of intraday returns and intraday and daily measures of market risk. Using data on the EUR/JPY exchange rate, we find a negative relationship between entropy and intraday Value-at-Risk, and also between entropy and intraday Expected Shortfall. This relationship is then used to forecast daily Value-at-Risk, using the entropy of the distribution of intraday returns as a predictor.

Operations Research, 2008

Information measures arise in many disciplines, including forecasting (where scoring rules are used to provide incentives for probability estimation), signal processing (where information gain is measured in physical units of relative entropy), decision analysis (where new information can lead to improved decisions), and finance (where investors optimize portfolios based on their private information and risk preferences). In this paper, we generalize the two most commonly used parametric families of scoring rules and demonstrate their relation to well-known generalized entropies and utility functions, shedding new light on the characteristics of alternative scoring rules as well as duality relationships between utility maximization and entropy minimization. In particular, we show that weighted forms of the pseudospherical and power scoring rules correspond exactly to measures of relative entropy (divergence) with convenient properties, and they also correspond exactly to the solutions of expected utility maximization problems in which a risk-averse decision maker whose utility function belongs to the linear-risk-tolerance family interacts with a risk-neutral betting opponent or a complete market for contingent claims in either a one-period or a two-period setting. When the market is incomplete, the corresponding problems of maximizing linear-risk-tolerance utility with the risk-tolerance coefficient are the duals of the problems of minimizing the pseudospherical or power divergence of order between the decision maker's subjective probability distribution and the set of risk-neutral distributions that support asset prices.

Journal of Economic Theory, 2014

We explore the intuitive idea that more information leads to greater dispersion of posterior beliefs about the expected state of the world. First, we show that two dispersion orders that have been widely used as informativeness criteria do not satisfy the desirable property of ordinality of states (OS), i.e., invariance to increasing monotone state transformations. Then, for the class of monotone information systems, we characterize the weakest information criteria that respect OS and imply the dispersion orders. Our characterizations consist of intuitive conditions on the joint distributions of signals and states. Because of OS, the information criteria induce the dispersion orders not only on the posterior expectations of states, but also of state utilities, under any strictly increasing vNM utility function.