You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
servers.List() pagination is extremely inefficient #3867
I was investigating remarkably bad performance in a component that runs github.com/gophercloud/gophercloud/v2/openstack/compute/v2/servers.List(), and the profiling showed that the problem is with how pagination is implemented inside of it. It unmarshals the response body no less than four times:
once into map[string]any during pagination.PageResultFrom() for use in generic NextPageURL/IsEmpty implementations (which do not end up being used)
once in ServersPage's custom NextPageURL implementation
once in ServersPage's custom IsEmpty implementation
once in ServersPage.Extract() to get the actual []servers.Server
While this is clearly suboptimal, usually it's not all that bad because most JSON responses from OpenStack are not all that large. But servers.List() contains server objects which not only have an extremely large number of attributes, some of those attributes are also genuinely huge (esp. userdata).
For my own usecase, I mitigated the problem by using a custom pagination implementation that bypasses the standard types from package pagination entirely. I don't think that it will be possible for the standard pagination implementation to be as efficient (because of how func Extract always does a separate unmarshal), but it might be possible to avoid the three separate unmarshals for NextPageURL and IsEmpty.
changed the title [-]servers.ListDetail() pagination is extremely inefficient[/-][+]servers.List() pagination is extremely inefficient[/+]on Jul 22, 2026
Measured on main with an EachPage handler that calls ExtractServers: each page is serialised three times and parsed four times, since IsEmpty and NextPageURL both go through ExtractInto, which re-encodes the body PageResultFrom already decoded. Reading servers and servers_links from the decoded map removes those two round trips, ExtractServers stays the only full decode. PR coming for the servers package; the generic decode in PageResultFrom stays since AllPages depends on it.
Activity
[-]servers.ListDetail() pagination is extremely inefficient[/-][+]servers.List() pagination is extremely inefficient[/+]blacksoxx commentedon Oct 11, 2026
Measured on main with an EachPage handler that calls ExtractServers: each page is serialised three times and parsed four times, since IsEmpty and NextPageURL both go through ExtractInto, which re-encodes the body PageResultFrom already decoded. Reading
serversandservers_linksfrom the decoded map removes those two round trips, ExtractServers stays the only full decode. PR coming for the servers package; the generic decode in PageResultFrom stays since AllPages depends on it.compute: stop re-decoding server list pages in IsEmpty and NextPageURL
compute: stop re-decoding server list pages in IsEmpty and NextPageURL