Skip to content

Pre-warm ramoth before applying load to it #92

@pnorman

Description

@pnorman

With cold caches it can be difficult to put load on ramoth without overloading it. The standard solution to this is pg_prewarm, but that requires 9.4.

Instead, this can be done with shell script as the user postgres

find "$(psql -d postgres -AXtqwc 'SHOW data_directory;')" -type f \
  | grep "$(psql -d osm2pgsql -AXtqwc 'SELECT pg_relation_filepath(r.oid) || '\''.*'\'' FROM pg_class r WHERE r.relname='\''planet_osm_polygon_index'\')" \
  | xargs -i cat {} > /dev/null

Replace osm2pgsql and planet_osm_polygon_index with appropriate db and index names. To do multiple indexes, run the command multiple times.

The shell script turns into a command like

find /var/lib/postgresql/9.5/main -type f | grep "base/5917088/8222095.*" | xargs -i cat {} > /dev/null

which then reads the /var/lib/postgresql/9.5/main/base/5917088/8222095, /var/lib/postgresql/9.5/main/base/5917088/8222095.1, /var/lib/postgresql/9.5/main/base/5917088/8222095.2, etc files.

This assumes that cat f > /dev/null reads the file, so it doesn't work on Solaris which uses mmap for cat.

With 9.4 or later, this is all done with CREATE EXTENSION pg_prewarm; SELECT pg_prewarm('planet_osm_polygon_index', 'prefetch');'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions