This is the second of a series of papers extending the 1+3 Covariant and Gauge-Invariant treatmen... more This is the second of a series of papers extending the 1+3 Covariant and Gauge-Invariant treatment of kinetic theory to an examination of Cosmic Microwave Background temperature anisotropies arising from inhomogeneities in the early universe. The first paper (Paper I) dealt with algebraic issues, representing anisotropies in a covariant and gauge invariant way by means of Projected Symmetric and Trace-Free tensors. Here we derive the mode form of the Integrated Boltzmann Equations, first, giving a covariant version of the standard derivation using the mode recursion relations, second, demonstrating the link to the the Multipole Divergence Equations and finally various analytic ways of solving the resulting equations are discussed. A general integral form of solution is obtained for the equations with Thomson scattering. The covariant Friedmann-Lemaître multipole form of the transport equations are found near tight-coupling using the covariant and gaugeinvariant generalization of the Peebles and Yu expansion in Thompson scattering time. The dispersion relations and damping scale are then obtained from the covariant approach. The equations are integrated to give the covariant and gauge-invariant equivalent of the canonical scalar sourced anisotropies in the K = 0 (flat background) case. We carry out a simple treatment of the matter dominated free-streaming projection, slow decoupling, and tight-coupling cases in covariant and gaugeinvariant theory, with the aim both giving a unified transparent derivation of this range of results and clarifying the formal connection between the usual approaches (for example Hu & Sugiyama) and the covariant and gauge-invariant like treatments for scalar perturbations (for example of Challinor & Lasenby).
2021 IEEE 24th International Conference on Information Fusion (FUSION), Nov 1, 2021
Mean-variance portfolio decisions that combine prediction and optimisation have been shown to hav... more Mean-variance portfolio decisions that combine prediction and optimisation have been shown to have poor empirical performance. Here, we consider the performance of various shrinkage methods by their efficient frontiers under different distributional assumptions to study the impact of reasonable departures from Normality. Namely, we investigate the impact of first-order auto-correlation, second-order auto-correlation, skewness, and excess kurtosis. We show that the shrinkage methods tend to re-scale the sample efficient frontier, which can change based on the nature of local perturbations from Normality. This re-scaling implies that the standard approach of comparing decision rules for a fixed level of risk aversion is problematic, and more so in a dynamic market setting. Our results suggest that comparing efficient frontiers has serious implications which oppose the prevailing thinking in the literature. Namely, that sample estimators out-perform Stein type estimators of the mean, and that improving the prediction of the covariance has greater importance than improving that of the means.
An agent-based model with interacting low frequency liquidity takers inter-mediated by high-frequ... more An agent-based model with interacting low frequency liquidity takers inter-mediated by high-frequency liquidity providers acting collectively as market makers can be used to provide realistic simulated price impact curves. This is possible when agent-based model interactions occur asynchronously via order matching using a matching engine in event time to replace sequential calendar time market clearing. Here the matching engine infrastructure has been modified to provide a continuous feed of order confirmations and updates as message streams in order to conform more closely to live trading environments. The resulting trade and quote message data from the simulations are then aggregated, calibrated and visualised. Various stylised facts are presented along with event visualisations and price impact curves. We argue that additional realism in modelling can be achieved with a small set of agent parameters and simple interaction rules once interactions are reactive, asynchronous and in event time. We argue that the reactive nature of market agents may be a fundamental property of financial markets and when accounted for can allow for parsimonious modelling without recourse to additional sources of noise.
The extent to which a matching engine can cloud the modelling of underlying order submission and ... more The extent to which a matching engine can cloud the modelling of underlying order submission and management processes in a financial market and the importance of this with regards to the statistical robustness of models and resulting inference remains an unanswered concern for market models. Here we consider a 10-variate Hawkes process with simple rules to simulate common order types which are submitted to a matching engine. Hawkes processes can be used to model the time and order of events, and how these events relate to each other. However, they provide a freedom with regards to implementation mechanics relating to the prices and volumes of injected orders. This allows us to consider a reference Hawkes model and two additional models which have rules that change the behaviour of limit orders. The resulting trade and quote data from the simulations are then calibrated and compared with the original order generating process to determine the extent with which implementation rules can distort model parameters. Evidence from validation and hypothesis tests suggest that the true model specification can be significantly distorted by market mechanics to the extent that the models are no longer statistically equivalent. It can be argued that the matching mechanism merely blurs the estimation and identification of the process and this is a triviality. However, we argue that this can lead to fundamental changes in the nature of decisions when using these types of models.
We investigate and extend the results of Golts and Jones that an alpha-weight angle resulting fro... more We investigate and extend the results of Golts and Jones that an alpha-weight angle resulting from unconstrained quadratic portfolio optimisations has an upper bound dependent on the condition number of the covariance matrix. This implies that better conditioned covariance matrices produce weights from unconstrained mean-variance optimisations that are better aligned with each assets expected return. We provide further clarity on the mathematical insights that relate the inequality between the α-weight angle and the condition number and extend the result to include portfolio optimisations with gearing constraints. We provide an extended family of robust optimisations that include the gearing constraints, and discuss their interpretation.
The Epps effect is key phenomenology relating to high frequency correlation dynamics in financial... more The Epps effect is key phenomenology relating to high frequency correlation dynamics in financial markets. We argue that it can be used to provide insight into whether tick data is best represented as samples from Brownian diffusions, or as samples from truly discrete events represented as connected point processes. We derive the Epps effect arising from asynchrony and provide a refined method to correct for the effect. We then propose three experiments which show how to discriminate between possible underlying representations. These in turn demonstrate how a simple Hawkes representation recovers phenomenology reported in the literature that cannot be recovered using a Brownian representation without additional ad hoc model complexity. However, complex ad hoc noise models built on Brownian motions cannot in general be discriminated relative to a Hawkes representation. Nevertheless, we argue that high frequency correlation dynamics are most faithfully recovered when tick data is represented as a web of interconnected discrete events rather than being samples from continuous Brownian diffusions even when combined with noise.
We consider the problem of fast time-series data clustering. Building on previous work modeling t... more We consider the problem of fast time-series data clustering. Building on previous work modeling the correlation-based Hamiltonian of spin variables we present an updated fast non-expensive Agglomerative Likelihood Clustering algorithm (ALC). The method replaces the optimized genetic algorithm based approach (f-SPC) with an agglomerative recursive merging framework inspired by previous work in Econophysics and Community Detection. The method is tested on noisy synthetic correlated time-series data-sets with built-in cluster structure to demonstrate that the algorithm produces meaningful non-trivial results. We apply it to time-series data-sets as large as 20,000 assets and we argue that ALC can reduce compute time costs and resource usage cost for large scale clustering for time-series applications while being serialized, and hence has no obvious parallelization requirement. The algorithm can be an effective choice for state-detection for online learning in a fast non-linear data environment because the algorithm requires no prior information about the number of clusters.
2021 IEEE 24th International Conference on Information Fusion (FUSION), 2021
Mean-variance portfolio decisions that combine prediction and optimisation have been shown to hav... more Mean-variance portfolio decisions that combine prediction and optimisation have been shown to have poor empirical performance. Here, we consider the performance of various shrinkage methods by their efficient frontiers under different distributional assumptions to study the impact of reasonable departures from Normality. Namely, we investigate the impact of first-order auto-correlation, second-order auto-correlation, skewness, and excess kurtosis. We show that the shrinkage methods tend to re-scale the sample efficient frontier, which can change based on the nature of local perturbations from Normality. This re-scaling implies that the standard approach of comparing decision rules for a fixed level of risk aversion is problematic, and more so in a dynamic market setting. Our results suggest that comparing efficient frontiers has serious implications which oppose the prevailing thinking in the literature. Namely, that sample estimators out-perform Stein type estimators of the mean, and that improving the prediction of the covariance has greater importance than improving that of the means.
A report submitted in fulfilment of the requirements for the degree of Honours in Statistical Sci... more A report submitted in fulfilment of the requirements for the degree of Honours in Statistical Science. The report acts as supporting material for the pre-print titled: "Revisiting the Epps effect using volume time averaging: An exercise in R"
Julia Pro code to aid reproducibility of the paper: Using the Epps effect to detect discrete data... more Julia Pro code to aid reproducibility of the paper: Using the Epps effect to detect discrete data generating processes. Detailed steps for reproducibility can be found in the READ.me document.<br>The code include several functions, scripts and test cases. Included are also the resulting plots for the paper and pre-computed results to speed up reproducibility.<br>The research focuses on empirically determining if correlations are an emerging property in high frequency finance.<br>The work was funded by the South African Statistical Association.<br>DOI for the code: 10.25375/uct.12315167<br>
We implement a systematic asset allocation model using the Historical Simulation with Flexible Pr... more We implement a systematic asset allocation model using the Historical Simulation with Flexible Probabilities (HS-FP) framework developed by Meucci. The HS-FP framework is a flexible non-parametric estimation approach that considers future asset class behavior to be conditional on time and market environments, and derives a forward looking distribution that is consistent with this view while remaining close as possible to the prior distribution. The framework derives the forward looking distribution by applying unequal time and state conditioned probabilities to historical observations of asset class returns. This is achieved using relative entropy to find estimates with the least distortion to the prior distribution. Here, we use the HS-FP framework on South African financial market data for asset allocation purposes; by estimating expected returns, correlations and volatilities that are better represented through the measured market cycle. We demonstrated a range of state variables...
We consider shared listings on two South African equity exchanges: the Johannesburg Stock Exchang... more We consider shared listings on two South African equity exchanges: the Johannesburg Stock Exchange (JSE) and the A2X Exchange. A2X is an alternative exchange that provides for both shared listings and new listings within the financial market ecosystem of South Africa. From a science perspective it provides the opportunity to compare markets trading similar shares, in a similar regulatory and economic environment, but with vastly different liquidity, costs and business models. A2X currently has competitive settlement and transaction pricing when compared to the JSE, but the JSE has deeper liquidity. In pursuit of an empirical understanding of how these differences relate to their respective price response dynamics, we compare the distributions and auto-correlations of returns on different time scales; we compare price impact and master curves; and we compare the cost of trading on each exchange. This allows us to empirically compare the two markets. We find that various stylised fact...
We revisit and demonstrate the Epps effect using two well-known non-parametric covariance estimat... more We revisit and demonstrate the Epps effect using two well-known non-parametric covariance estimators; the Malliavin and Mancino (MM), and Hayashi and Yoshida (HY) estimators. We show the existence of the Epps effect in the top 10 stocks from the Johannesburg Stock Exchange (JSE) by various methods of aggregating Trade and Quote (TAQ) data. Concretely, we compare calendar time sampling with two volume time sampling methods: asset intrinsic volume time averaging, and volume time averaging synchronised in volume time across assets relative to the least and most liquid asset clocks. We reaffirm the argument made in much of the literature that the MM estimator is more representative of trade time reality because it does not over-estimate short-term correlations in an asynchronous event driven world. We confirm well known market phenomenology with the aim of providing some standardised R based simulation tools.
We use an adversarial expert based online learning algorithm to learn the optimal parameters requ... more We use an adversarial expert based online learning algorithm to learn the optimal parameters required to maximise wealth trading zero-cost portfolio strategies. The learning algorithm is used to determine the dynamics of a large population of technical trading strategies that can survive historical back-testing as well as form an overall aggregated portfolio trading strategy from the set of underlying trading strategies implemented on daily and intraday Johannesburg Stock Exchange data. The resulting population time-series are investigated using unsupervised learning for dimensionality reduction and visualisation. A key contribution is that the overall aggregated trading strategies are tested for statistical arbitrage using a novel hypothesis test, on both daily sampled and intraday time-scales. The (low frequency) daily sampled strategies fail the arbitrage tests after costs, while the (high frequency) intraday sampled strategies are not falsified as statistical arbitrages after co...
Journal of Statistical Mechanics: Theory and Experiment, 2021
We consider the problem of fast time-series data clustering. Building on previous work modeling, ... more We consider the problem of fast time-series data clustering. Building on previous work modeling, the correlation-based Hamiltonian of spin variables we present an updated fast non-expensive agglomerative likelihood clustering algorithm (ALC). The method replaces the optimized genetic algorithm based approach (f-SPC) with an agglomerative recursive merging framework inspired by previous work in econophysics and community detection. The method is tested on noisy synthetic correlated time-series datasets with a built-in cluster structure to demonstrate that the algorithm produces meaningful non-trivial results. We apply it to time-series datasets as large as 20 000 assets and we argue that ALC can reduce computation time costs and resource usage costs for large scale clustering for time-series applications while being serialized, and hence has no obvious parallelization requirement. The algorithm can be an effective choice for state-detection for online learning in a fast non-linear da...
We replicate the contested calibration of the Farmer and Joshi [1] agent based model of financial... more We replicate the contested calibration of the Farmer and Joshi [1] agent based model of financial markets using a genetic algorithm and a Nelder-Mead with threshold accepting algorithm following Fabretti [2]. The novelty of the Farmer-Joshi model is that the dynamics are driven by trade entry and exit thresholds alone. We recover the known claim that some important stylized facts observed in financial markets cannot be easily found under calibration – in particular those relating to the auto-correlations in the absolute values of the price fluctuations, and sufficient kurtosis. However, rather than concerns relating to the calibration method, what is novel here is that we extended the Farmer-Joshi model to include agent adaptation using an Brock and Hommes [3] approach to strategy fitness based on trading strategy profitability. We call this an adaptive Farmer-Joshi model: the model allows trading agents to switch between strategies by favouring strategies that have been more profit...
We use an adversarial expert based online learning algorithm to learn the optimal parameters requ... more We use an adversarial expert based online learning algorithm to learn the optimal parameters required to maximise wealth trading zero-cost portfolio strategies. The learning algorithm is used to determine the relative population dynamics of technical trading strategies that can survive historical back-testing as well as form an overall aggregated portfolio trading strategy from the set of underlying trading strategies implemented on daily and intraday Johannesburg Stock Exchange data. The resulting population time-series are investigated using unsupervised learning for dimensionality reduction and visualisation. A key contribution is that the overall aggregated trading strategies are tested for statistical arbitrage using a novel hypothesis test proposed by Jarrow et al. on both daily sampled and intraday time-scales. The (low frequency) daily sampled strategies fail the arbitrage tests after costs, while the (high frequency) intraday sampled strategies are not falsified as statisti...
This is the second of a series of papers extending the 1+3 Covariant and Gauge-Invariant treatmen... more This is the second of a series of papers extending the 1+3 Covariant and Gauge-Invariant treatment of kinetic theory to an examination of Cosmic Microwave Background temperature anisotropies arising from inhomogeneities in the early universe. The first paper (Paper I) dealt with algebraic issues, representing anisotropies in a covariant and gauge invariant way by means of Projected Symmetric and Trace-Free tensors. Here we derive the mode form of the Integrated Boltzmann Equations, first, giving a covariant version of the standard derivation using the mode recursion relations, second, demonstrating the link to the the Multipole Divergence Equations and finally various analytic ways of solving the resulting equations are discussed. A general integral form of solution is obtained for the equations with Thomson scattering. The covariant Friedmann-Lemaître multipole form of the transport equations are found near tight-coupling using the covariant and gaugeinvariant generalization of the Peebles and Yu expansion in Thompson scattering time. The dispersion relations and damping scale are then obtained from the covariant approach. The equations are integrated to give the covariant and gauge-invariant equivalent of the canonical scalar sourced anisotropies in the K = 0 (flat background) case. We carry out a simple treatment of the matter dominated free-streaming projection, slow decoupling, and tight-coupling cases in covariant and gaugeinvariant theory, with the aim both giving a unified transparent derivation of this range of results and clarifying the formal connection between the usual approaches (for example Hu & Sugiyama) and the covariant and gauge-invariant like treatments for scalar perturbations (for example of Challinor & Lasenby).
We consider the problem of fast time-series data clustering. Building on previous work modeling t... more We consider the problem of fast time-series data clustering. Building on previous work modeling the correlation-based Hamiltonian of spin variables we present a fast non-expensive agglomerative algorithm. The method is tested on synthetic correlated time-series and noisy synthetic data-sets with built-in cluster structure to demonstrate that the algorithm produces meaningful non-trivial results. We argue that ASPC can reduce compute time costs and resource usage cost for large scale clustering while being serialized and hence has no obvious parallelization requirement. The algorithm can be an effective choice for state-detection for online learning in a fast non-linear data environment because the algorithm requires no prior information about the number of clusters.
We implement and test kernel averaging Non-Uniform Fast-Fourier Transform (NUFFT) methods to enha... more We implement and test kernel averaging Non-Uniform Fast-Fourier Transform (NUFFT) methods to enhance the performance of correlation and covariance estimation on asynchronously sampled event-data using the Malliavin-Mancino Fourier estimator. The methods are benchmarked for Dirichlet and Fejér Fourier basis kernels. We consider test cases formed from Geometric Brownian motions to replicate synchronous and asynchronous data for benchmarking purposes. We consider three standard averaging kernels to convolve the event-data for synchronisation via over-sampling for use with the Fast Fourier Transform (FFT): the Gaussian kernel, the Kaiser-Bessel kernel, and the exponential of semi-circle kernel. First, this allows us to demonstrate the performance of the estimator with different combinations of basis kernels and averaging kernels. Second, we investigate and compare the impact of the averaging scales explicit in each averaging kernel and its relationship between the time-scale averaging implicit in the Malliavin-Mancino estimator. Third, we demonstrate the relationship between time-scale averaging based on the number of Fourier coefficients used in the estimator to a theoretical model of the Epps effect. We briefly demonstrate the methods on Trade-and-Quote (TAQ) data from the Johannesburg Stock Exchange to make an initial visualisation of the correlation dynamics for various time-scales under market microstructure.
This is the second of a series of papers extending the 1+3 Covariant and Gauge-Invariant treatmen... more This is the second of a series of papers extending the 1+3 Covariant and Gauge-Invariant treatment of kinetic theory to an examination of Cosmic Microwave Background temperature anisotropies arising from inhomogeneities in the early universe. The first paper (Paper I) dealt with algebraic issues, representing anisotropies in a covariant and gauge invariant way by means of Projected Symmetric and Trace-Free tensors. Here we derive the mode form of the Integrated Boltzmann Equations, first, giving a covariant version of the standard derivation using the mode recursion relations, second, demonstrating the link to the the Multipole Divergence Equations and finally various analytic ways of solving the resulting equations are discussed. A general integral form of solution is obtained for the equations with Thomson scattering. The covariant Friedmann-Lemaître multipole form of the transport equations are found near tight-coupling using the covariant and gaugeinvariant generalization of the Peebles and Yu expansion in Thompson scattering time. The dispersion relations and damping scale are then obtained from the covariant approach. The equations are integrated to give the covariant and gauge-invariant equivalent of the canonical scalar sourced anisotropies in the K = 0 (flat background) case. We carry out a simple treatment of the matter dominated free-streaming projection, slow decoupling, and tight-coupling cases in covariant and gaugeinvariant theory, with the aim both giving a unified transparent derivation of this range of results and clarifying the formal connection between the usual approaches (for example Hu & Sugiyama) and the covariant and gauge-invariant like treatments for scalar perturbations (for example of Challinor & Lasenby).
2021 IEEE 24th International Conference on Information Fusion (FUSION), Nov 1, 2021
Mean-variance portfolio decisions that combine prediction and optimisation have been shown to hav... more Mean-variance portfolio decisions that combine prediction and optimisation have been shown to have poor empirical performance. Here, we consider the performance of various shrinkage methods by their efficient frontiers under different distributional assumptions to study the impact of reasonable departures from Normality. Namely, we investigate the impact of first-order auto-correlation, second-order auto-correlation, skewness, and excess kurtosis. We show that the shrinkage methods tend to re-scale the sample efficient frontier, which can change based on the nature of local perturbations from Normality. This re-scaling implies that the standard approach of comparing decision rules for a fixed level of risk aversion is problematic, and more so in a dynamic market setting. Our results suggest that comparing efficient frontiers has serious implications which oppose the prevailing thinking in the literature. Namely, that sample estimators out-perform Stein type estimators of the mean, and that improving the prediction of the covariance has greater importance than improving that of the means.
An agent-based model with interacting low frequency liquidity takers inter-mediated by high-frequ... more An agent-based model with interacting low frequency liquidity takers inter-mediated by high-frequency liquidity providers acting collectively as market makers can be used to provide realistic simulated price impact curves. This is possible when agent-based model interactions occur asynchronously via order matching using a matching engine in event time to replace sequential calendar time market clearing. Here the matching engine infrastructure has been modified to provide a continuous feed of order confirmations and updates as message streams in order to conform more closely to live trading environments. The resulting trade and quote message data from the simulations are then aggregated, calibrated and visualised. Various stylised facts are presented along with event visualisations and price impact curves. We argue that additional realism in modelling can be achieved with a small set of agent parameters and simple interaction rules once interactions are reactive, asynchronous and in event time. We argue that the reactive nature of market agents may be a fundamental property of financial markets and when accounted for can allow for parsimonious modelling without recourse to additional sources of noise.
The extent to which a matching engine can cloud the modelling of underlying order submission and ... more The extent to which a matching engine can cloud the modelling of underlying order submission and management processes in a financial market and the importance of this with regards to the statistical robustness of models and resulting inference remains an unanswered concern for market models. Here we consider a 10-variate Hawkes process with simple rules to simulate common order types which are submitted to a matching engine. Hawkes processes can be used to model the time and order of events, and how these events relate to each other. However, they provide a freedom with regards to implementation mechanics relating to the prices and volumes of injected orders. This allows us to consider a reference Hawkes model and two additional models which have rules that change the behaviour of limit orders. The resulting trade and quote data from the simulations are then calibrated and compared with the original order generating process to determine the extent with which implementation rules can distort model parameters. Evidence from validation and hypothesis tests suggest that the true model specification can be significantly distorted by market mechanics to the extent that the models are no longer statistically equivalent. It can be argued that the matching mechanism merely blurs the estimation and identification of the process and this is a triviality. However, we argue that this can lead to fundamental changes in the nature of decisions when using these types of models.
We investigate and extend the results of Golts and Jones that an alpha-weight angle resulting fro... more We investigate and extend the results of Golts and Jones that an alpha-weight angle resulting from unconstrained quadratic portfolio optimisations has an upper bound dependent on the condition number of the covariance matrix. This implies that better conditioned covariance matrices produce weights from unconstrained mean-variance optimisations that are better aligned with each assets expected return. We provide further clarity on the mathematical insights that relate the inequality between the α-weight angle and the condition number and extend the result to include portfolio optimisations with gearing constraints. We provide an extended family of robust optimisations that include the gearing constraints, and discuss their interpretation.
The Epps effect is key phenomenology relating to high frequency correlation dynamics in financial... more The Epps effect is key phenomenology relating to high frequency correlation dynamics in financial markets. We argue that it can be used to provide insight into whether tick data is best represented as samples from Brownian diffusions, or as samples from truly discrete events represented as connected point processes. We derive the Epps effect arising from asynchrony and provide a refined method to correct for the effect. We then propose three experiments which show how to discriminate between possible underlying representations. These in turn demonstrate how a simple Hawkes representation recovers phenomenology reported in the literature that cannot be recovered using a Brownian representation without additional ad hoc model complexity. However, complex ad hoc noise models built on Brownian motions cannot in general be discriminated relative to a Hawkes representation. Nevertheless, we argue that high frequency correlation dynamics are most faithfully recovered when tick data is represented as a web of interconnected discrete events rather than being samples from continuous Brownian diffusions even when combined with noise.
We consider the problem of fast time-series data clustering. Building on previous work modeling t... more We consider the problem of fast time-series data clustering. Building on previous work modeling the correlation-based Hamiltonian of spin variables we present an updated fast non-expensive Agglomerative Likelihood Clustering algorithm (ALC). The method replaces the optimized genetic algorithm based approach (f-SPC) with an agglomerative recursive merging framework inspired by previous work in Econophysics and Community Detection. The method is tested on noisy synthetic correlated time-series data-sets with built-in cluster structure to demonstrate that the algorithm produces meaningful non-trivial results. We apply it to time-series data-sets as large as 20,000 assets and we argue that ALC can reduce compute time costs and resource usage cost for large scale clustering for time-series applications while being serialized, and hence has no obvious parallelization requirement. The algorithm can be an effective choice for state-detection for online learning in a fast non-linear data environment because the algorithm requires no prior information about the number of clusters.
2021 IEEE 24th International Conference on Information Fusion (FUSION), 2021
Mean-variance portfolio decisions that combine prediction and optimisation have been shown to hav... more Mean-variance portfolio decisions that combine prediction and optimisation have been shown to have poor empirical performance. Here, we consider the performance of various shrinkage methods by their efficient frontiers under different distributional assumptions to study the impact of reasonable departures from Normality. Namely, we investigate the impact of first-order auto-correlation, second-order auto-correlation, skewness, and excess kurtosis. We show that the shrinkage methods tend to re-scale the sample efficient frontier, which can change based on the nature of local perturbations from Normality. This re-scaling implies that the standard approach of comparing decision rules for a fixed level of risk aversion is problematic, and more so in a dynamic market setting. Our results suggest that comparing efficient frontiers has serious implications which oppose the prevailing thinking in the literature. Namely, that sample estimators out-perform Stein type estimators of the mean, and that improving the prediction of the covariance has greater importance than improving that of the means.
A report submitted in fulfilment of the requirements for the degree of Honours in Statistical Sci... more A report submitted in fulfilment of the requirements for the degree of Honours in Statistical Science. The report acts as supporting material for the pre-print titled: "Revisiting the Epps effect using volume time averaging: An exercise in R"
Julia Pro code to aid reproducibility of the paper: Using the Epps effect to detect discrete data... more Julia Pro code to aid reproducibility of the paper: Using the Epps effect to detect discrete data generating processes. Detailed steps for reproducibility can be found in the READ.me document.<br>The code include several functions, scripts and test cases. Included are also the resulting plots for the paper and pre-computed results to speed up reproducibility.<br>The research focuses on empirically determining if correlations are an emerging property in high frequency finance.<br>The work was funded by the South African Statistical Association.<br>DOI for the code: 10.25375/uct.12315167<br>
We implement a systematic asset allocation model using the Historical Simulation with Flexible Pr... more We implement a systematic asset allocation model using the Historical Simulation with Flexible Probabilities (HS-FP) framework developed by Meucci. The HS-FP framework is a flexible non-parametric estimation approach that considers future asset class behavior to be conditional on time and market environments, and derives a forward looking distribution that is consistent with this view while remaining close as possible to the prior distribution. The framework derives the forward looking distribution by applying unequal time and state conditioned probabilities to historical observations of asset class returns. This is achieved using relative entropy to find estimates with the least distortion to the prior distribution. Here, we use the HS-FP framework on South African financial market data for asset allocation purposes; by estimating expected returns, correlations and volatilities that are better represented through the measured market cycle. We demonstrated a range of state variables...
We consider shared listings on two South African equity exchanges: the Johannesburg Stock Exchang... more We consider shared listings on two South African equity exchanges: the Johannesburg Stock Exchange (JSE) and the A2X Exchange. A2X is an alternative exchange that provides for both shared listings and new listings within the financial market ecosystem of South Africa. From a science perspective it provides the opportunity to compare markets trading similar shares, in a similar regulatory and economic environment, but with vastly different liquidity, costs and business models. A2X currently has competitive settlement and transaction pricing when compared to the JSE, but the JSE has deeper liquidity. In pursuit of an empirical understanding of how these differences relate to their respective price response dynamics, we compare the distributions and auto-correlations of returns on different time scales; we compare price impact and master curves; and we compare the cost of trading on each exchange. This allows us to empirically compare the two markets. We find that various stylised fact...
We revisit and demonstrate the Epps effect using two well-known non-parametric covariance estimat... more We revisit and demonstrate the Epps effect using two well-known non-parametric covariance estimators; the Malliavin and Mancino (MM), and Hayashi and Yoshida (HY) estimators. We show the existence of the Epps effect in the top 10 stocks from the Johannesburg Stock Exchange (JSE) by various methods of aggregating Trade and Quote (TAQ) data. Concretely, we compare calendar time sampling with two volume time sampling methods: asset intrinsic volume time averaging, and volume time averaging synchronised in volume time across assets relative to the least and most liquid asset clocks. We reaffirm the argument made in much of the literature that the MM estimator is more representative of trade time reality because it does not over-estimate short-term correlations in an asynchronous event driven world. We confirm well known market phenomenology with the aim of providing some standardised R based simulation tools.
We use an adversarial expert based online learning algorithm to learn the optimal parameters requ... more We use an adversarial expert based online learning algorithm to learn the optimal parameters required to maximise wealth trading zero-cost portfolio strategies. The learning algorithm is used to determine the dynamics of a large population of technical trading strategies that can survive historical back-testing as well as form an overall aggregated portfolio trading strategy from the set of underlying trading strategies implemented on daily and intraday Johannesburg Stock Exchange data. The resulting population time-series are investigated using unsupervised learning for dimensionality reduction and visualisation. A key contribution is that the overall aggregated trading strategies are tested for statistical arbitrage using a novel hypothesis test, on both daily sampled and intraday time-scales. The (low frequency) daily sampled strategies fail the arbitrage tests after costs, while the (high frequency) intraday sampled strategies are not falsified as statistical arbitrages after co...
Journal of Statistical Mechanics: Theory and Experiment, 2021
We consider the problem of fast time-series data clustering. Building on previous work modeling, ... more We consider the problem of fast time-series data clustering. Building on previous work modeling, the correlation-based Hamiltonian of spin variables we present an updated fast non-expensive agglomerative likelihood clustering algorithm (ALC). The method replaces the optimized genetic algorithm based approach (f-SPC) with an agglomerative recursive merging framework inspired by previous work in econophysics and community detection. The method is tested on noisy synthetic correlated time-series datasets with a built-in cluster structure to demonstrate that the algorithm produces meaningful non-trivial results. We apply it to time-series datasets as large as 20 000 assets and we argue that ALC can reduce computation time costs and resource usage costs for large scale clustering for time-series applications while being serialized, and hence has no obvious parallelization requirement. The algorithm can be an effective choice for state-detection for online learning in a fast non-linear da...
We replicate the contested calibration of the Farmer and Joshi [1] agent based model of financial... more We replicate the contested calibration of the Farmer and Joshi [1] agent based model of financial markets using a genetic algorithm and a Nelder-Mead with threshold accepting algorithm following Fabretti [2]. The novelty of the Farmer-Joshi model is that the dynamics are driven by trade entry and exit thresholds alone. We recover the known claim that some important stylized facts observed in financial markets cannot be easily found under calibration – in particular those relating to the auto-correlations in the absolute values of the price fluctuations, and sufficient kurtosis. However, rather than concerns relating to the calibration method, what is novel here is that we extended the Farmer-Joshi model to include agent adaptation using an Brock and Hommes [3] approach to strategy fitness based on trading strategy profitability. We call this an adaptive Farmer-Joshi model: the model allows trading agents to switch between strategies by favouring strategies that have been more profit...
We use an adversarial expert based online learning algorithm to learn the optimal parameters requ... more We use an adversarial expert based online learning algorithm to learn the optimal parameters required to maximise wealth trading zero-cost portfolio strategies. The learning algorithm is used to determine the relative population dynamics of technical trading strategies that can survive historical back-testing as well as form an overall aggregated portfolio trading strategy from the set of underlying trading strategies implemented on daily and intraday Johannesburg Stock Exchange data. The resulting population time-series are investigated using unsupervised learning for dimensionality reduction and visualisation. A key contribution is that the overall aggregated trading strategies are tested for statistical arbitrage using a novel hypothesis test proposed by Jarrow et al. on both daily sampled and intraday time-scales. The (low frequency) daily sampled strategies fail the arbitrage tests after costs, while the (high frequency) intraday sampled strategies are not falsified as statisti...
This is the second of a series of papers extending the 1+3 Covariant and Gauge-Invariant treatmen... more This is the second of a series of papers extending the 1+3 Covariant and Gauge-Invariant treatment of kinetic theory to an examination of Cosmic Microwave Background temperature anisotropies arising from inhomogeneities in the early universe. The first paper (Paper I) dealt with algebraic issues, representing anisotropies in a covariant and gauge invariant way by means of Projected Symmetric and Trace-Free tensors. Here we derive the mode form of the Integrated Boltzmann Equations, first, giving a covariant version of the standard derivation using the mode recursion relations, second, demonstrating the link to the the Multipole Divergence Equations and finally various analytic ways of solving the resulting equations are discussed. A general integral form of solution is obtained for the equations with Thomson scattering. The covariant Friedmann-Lemaître multipole form of the transport equations are found near tight-coupling using the covariant and gaugeinvariant generalization of the Peebles and Yu expansion in Thompson scattering time. The dispersion relations and damping scale are then obtained from the covariant approach. The equations are integrated to give the covariant and gauge-invariant equivalent of the canonical scalar sourced anisotropies in the K = 0 (flat background) case. We carry out a simple treatment of the matter dominated free-streaming projection, slow decoupling, and tight-coupling cases in covariant and gaugeinvariant theory, with the aim both giving a unified transparent derivation of this range of results and clarifying the formal connection between the usual approaches (for example Hu & Sugiyama) and the covariant and gauge-invariant like treatments for scalar perturbations (for example of Challinor & Lasenby).
We consider the problem of fast time-series data clustering. Building on previous work modeling t... more We consider the problem of fast time-series data clustering. Building on previous work modeling the correlation-based Hamiltonian of spin variables we present a fast non-expensive agglomerative algorithm. The method is tested on synthetic correlated time-series and noisy synthetic data-sets with built-in cluster structure to demonstrate that the algorithm produces meaningful non-trivial results. We argue that ASPC can reduce compute time costs and resource usage cost for large scale clustering while being serialized and hence has no obvious parallelization requirement. The algorithm can be an effective choice for state-detection for online learning in a fast non-linear data environment because the algorithm requires no prior information about the number of clusters.
We implement and test kernel averaging Non-Uniform Fast-Fourier Transform (NUFFT) methods to enha... more We implement and test kernel averaging Non-Uniform Fast-Fourier Transform (NUFFT) methods to enhance the performance of correlation and covariance estimation on asynchronously sampled event-data using the Malliavin-Mancino Fourier estimator. The methods are benchmarked for Dirichlet and Fejér Fourier basis kernels. We consider test cases formed from Geometric Brownian motions to replicate synchronous and asynchronous data for benchmarking purposes. We consider three standard averaging kernels to convolve the event-data for synchronisation via over-sampling for use with the Fast Fourier Transform (FFT): the Gaussian kernel, the Kaiser-Bessel kernel, and the exponential of semi-circle kernel. First, this allows us to demonstrate the performance of the estimator with different combinations of basis kernels and averaging kernels. Second, we investigate and compare the impact of the averaging scales explicit in each averaging kernel and its relationship between the time-scale averaging implicit in the Malliavin-Mancino estimator. Third, we demonstrate the relationship between time-scale averaging based on the number of Fourier coefficients used in the estimator to a theoretical model of the Epps effect. We briefly demonstrate the methods on Trade-and-Quote (TAQ) data from the Johannesburg Stock Exchange to make an initial visualisation of the correlation dynamics for various time-scales under market microstructure.
Uploads
Papers by Tim Gebbie