Hunting Performance in Python Code – Part 2. Measuring Memory Consumption

May 18, 2017August 25, 2017 pythonrinf2 Comments

In this post I will talk about some tools that can help us solve a painful problem in Python, especially when using PyPy: memory consumption.

Why are we concerned with this in the first place? Why don’t we care only about performance? The answer to these questions is rather complex, but I’ll summarize it.

PyPy is an alternative Python interpreter, that features some great advantages over CPython: speed (through it’s Just in Time compiler), compatibility (it is almost a drop in replacement of CPython) and concurrency (using stackless and greenlets).

One downside of PyPy is that in general it uses more memory than CPython, due to it’s JIT and garbage collector implementation. Nevertheless, in some cases, it is able to use less memory than CPython.

We will see next how you can measure the amount of memory used by your application.

Hunting Performance in Python Code – Part 1. Environment Setup

May 15, 2017August 25, 2017 pythonrinf3 Comments

This is the first post in the series “Hunting Performance in Python Code”. Through each post I’ll present some of the tools and profilers that exists for Python code and how each of them helps you to better find bottlenecks both in frontend (Python scripts) and/or in the backend (Python interpreter).

Enabling Profile Guided Optimizations for PyPy

May 12, 2017June 9, 2017 pythonrinf2 Comments

PyPy, compared to CPython relies more on achieving speed-up by “jitting” code as often as possible, rather than rely on its interpreter. However, jitting is not always an option, or at least not entirely. A good improvement for CPython, that we think might benefit PyPy as well, without impacting the JIT performance is Profile Guided Optimization (PGO or profopt).

I thank the PyPy developer community for their patience, kind advice and constant feedback they gave me in #pypy IRC or through email, which helped me to make this possible, especially to Carl Friedrich Bolz-Tereick and Armin Rigo.

PyPy vs CPython Performance on OpenStack Neutron

May 4, 2017May 21, 2017 pythonfilesLeave a comment

This post contains details about a performance comparison between PyPy and CPython, when applied to OpenStack Neutron. The steps did to switch from CPython to PyPy are described in my post here.Read More »

Enabling PyPy as support for OpenStack Neutron

May 4, 2017May 21, 2017 pythonfiles5 Comments

This is a guide to deploy OpenStack Neutron with PyPy support. The underlying purpose was to determine whether using PyPy instead of CPython works for the networking service, and whether we would notice any improvements and/or drawbacks. The performance results of this can be read in my post here.

Python Files

Random thoughts about Python

Month: May 2017