My ”default” postgresql.

conf file,
step by step

Ilya Kosmodemiansky
[email protected]
 Before we start

• 269 settings in version 10

Why this talk
• 314 so far in version 12
• Settings in postgresql.conf are to be change manually
• postgresql.auto.conf
• Linux can becommon
is a most changed OS foronly through ALTER
databases
SYSTEM
• DBAs often run into IO problems
• pg settings view combines everything together
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 pg settings

postgres=# \x
Expanded display is on.
postgres=# select * from pg_settings where name ~ ’checkpoint_timeout’;
-[ RECORD 1 ]---+---------------------------------------------------------

Why this talk

name | checkpoint_timeout
setting | 3600
unit | s
category | Write-Ahead Log / Checkpoints
short_desc | Sets the maximum time between automatic WAL checkpoints.
extra_desc |
context | sighup
vartype | integer
source • Linuxfile
| configuration is a most common OS for databases
min_val | 30
max_val | 86400 • DBAs often run into IO problems
enumvals |
boot_val | 300 • Most of the information on topic is written by kerneldevelopers
reset_val | 3600
sourcefile (for kernel developers) or is checklist-style
| /etc/postgresql/10/main/postgresql.conf
sourceline | 208
pending_restart | f • Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 pg settings: context

Why this talk

postgres=# select distinct(context) from pg_settings ;
context
-------------------
postmaster
superuser-backend
user
internal
backend • Linux is a most common OS for databases
sighup
superuser
(7 rows) • DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 postgresql.conf

Why the
• Plase do not change thisorder
talk of the settings when you edit
them manually

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 postgresql.conf

Why the
• Plase do not change thisorder
talk of the settings when you edit
them manually
• postgresql.conf supports includes
• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 postgresql.conf

Why the
• Plase do not change thisorder
talk of the settings when you edit
them manually
• postgresql.conf supports includes
• Always check pg• Linux is a mostifcommon
settings OS for databases
you doubt...
• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 postgresql.conf

Why the
• Plase do not change thisorder
talk of the settings when you edit
them manually
• postgresql.conf supports includes
• Always check pg• Linux is a mostifcommon
settings OS for databases
you doubt...
• And off we go • DBAs often run into IO problems
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 listen addresses

Why this talk

• * or 127.0.0.1

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 listen addresses

Why this talk

• * or 127.0.0.1
• 127.0.0.1 is OK, when pgbouncer is used
• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 listen addresses

Why this talk

• * or 127.0.0.1
• 127.0.0.1 is OK, when pgbouncer is used
• Your database must
• Linuxbe firewall
is a most protected
common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 max connections

Why
• Client connection thisPostgres
cause talk to spawn a ”heavy”
Unix-Process

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 max connections

Why
• Client connection thisPostgres
cause talk to spawn a ”heavy”
Unix-Process
• Thats why things like max connections = 1000 will never work
• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 max connections

Why
• Client connection thisPostgres
cause talk to spawn a ”heavy”
Unix-Process
• Thats why things like max connections = 1000 will never work
• A much better idea:
• Linuxmax
is a most common OS for=
connections databases
100 or 200 and really
small pool sizes• inDBAs
pgbouncer orIOanother
often run into problems connection pooler

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 superuser reserved connections

Why this talk

• When all of max connections are utilized, DBA needs to
connect to a database server in order to troubleshoot such
situation
• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 superuser reserved connections

Why this talk

• When all of max connections are utilized, DBA needs to
connect to a database server in order to troubleshoot such
situation
• Linux is a most common OS for databases
• Should be at least 5, better 10
• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 tcp keepalives idle

Why this talk

• If network is unstable, 5 seconds can really help

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 tcp keepalives idle

Why this talk

• If network is unstable, 5 seconds can really help
• tcp keepalives interval = 1
• tcp keepalives count
• Linux=
is a 5
most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 shared buffers

• Rule of Thumb:Why
25% this talk
of RAM

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 shared buffers

• Rule of Thumb:Why25% this talk

of RAM
• But to use 16/32/64Gb of shared buffers efficiently, fast
discs are required
• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 shared buffers

• Rule of Thumb:Why 25% this of RAMtalk

• But to use 16/32/64Gb of shared buffers efficiently, fast
discs are required
• If the database •is Linux is a most common
definitely smallerOSthanfor databases
RAM, 75% of RAM
for shared buffers canoften
• DBAs alsorunwork
into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 huge pages

• Rule of thumb: Why

when this
theretalk
are 8-16Gb shared buffers, using of
Huge Pages is recommended
• huge pages = on
• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 huge pages

• Rule of thumb: Why

when this
theretalk
are 8-16Gb shared buffers, using of
Huge Pages is recommended
• huge pages = on
• Huge Pages should
• Linux
beis first
a most enabled
common OSin
for kernel
databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 huge pages

• Rule of thumb: Why

when this
theretalk
are 8-16Gb shared buffers, using of
Huge Pages is recommended
• huge pages = on
• Huge Pages should
• Linux
beis first
a most enabled
common OSin for kernel
databases

• vm.nr overcommit
• DBAs often run intoand
hugepages IO problems
vm.nr hugepages
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Linux Memory allocation

CPU L1
Virtual addressing

Why this talk

MMU TLB
Translation

L2• Linux is a mostL3

common OS for databases

• DBAs often run into IO problems

Memory page
table
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style
Physical addressing
• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 work mem

• RAM per process,Why this talk

Postgres workers use this RAM for sorting,
hash joins etc.

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 work mem

• RAM per process,Why this talk

Postgres workers use this RAM for sorting,
hash joins etc.
• 128Mb is a good starting point
• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 work mem

• RAM per process,Why this talk

Postgres workers use this RAM for sorting,
hash joins etc.
• 128Mb is a good starting point
• To high setting •could
Linux iscause
a most common
OOM OS for databases
• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 work mem

• RAM per process,Why this talk

Postgres workers use this RAM for sorting,
hash joins etc.
• 128Mb is a good starting point
• To high setting •could
Linux iscause
a most common
OOM OS for databases
• Could be individually
• DBAs configured
often run into IOfor
problems
each session
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 maintenance work mem

• Same as work mem but for superuser connections

Why this talk

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 maintenance work mem

• Same as work mem but for superuser connections

Why this talk
• 256-512Mb, if there is enough RAM

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 maintenance work mem

• Same as work mem but for superuser connections

Why this talk
• 256-512Mb, if there is enough RAM
• Could be quite helpful for CREATE INDEX
CONCURRENTLY • Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 maintenance work mem

• Same as work mem but for superuser connections

Why this talk
• 256-512Mb, if there is enough RAM
• Could be quite helpful for CREATE INDEX
CONCURRENTLY • Linux is a most common OS for databases
• autovacuum work mem is a part of maintenance work mem,
• DBAs often run into IO problems
can be smaller
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Settings fro manual vacuum

Why
• vacuum cost delay = this
0 talk
• vacuum cost page hit = 0
• vacuum cost page miss = 10
• Linux
• vacuum cost page is a most
dirty = 10 common OS for databases

• DBAs
• vacuum cost limit often run into IO problems
= 100
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Write Ahead Log

Why this talk

• wal level = replica

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Write Ahead Log

Why this talk

• wal level = replica
• checkpoint timeout = 60min, if it is by given recovery target
acceptable, could gain performance improvement
• max wal size = • 16GB
Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Write Ahead Log

Why this talk

• wal level = replica
• checkpoint timeout = 60min, if it is by given recovery target
acceptable, could gain performance improvement
• max wal size = • 16GB
Linux is a most common OS for databases

• checkpoint completion
• DBAs often run into
target =IO0.9
problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 bgwriter

• Background Writer helps Checkpointer to send unused dirty

pages to disk
Why this talk

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 bgwriter

• Background Writer helps Checkpointer to send unused dirty

pages to disk
Why this talk
• Regret to say, it is not the best part of PostgreSQL codebase

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 bgwriter

• Background Writer helps Checkpointer to send unused dirty

pages to disk
Why this talk
• Regret to say, it is not the best part of PostgreSQL codebase
• All settings to maximum:
• Linux is a most common OS for databases
I bgwriter delay = 10ms
I bgwriter lru• maxpages
DBAs often run
= into IO problems
1000
I bgwriter lru multiplier = 10.0
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 effective io concurrency

• 1 by default, which enables prefetch

Why this talk

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 effective io concurrency

• 1 by default, which enables prefetch

Why this talk
• Documentation recommends higher values for storages with
high parallelism capabilities

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 effective io concurrency

• 1 by default, which enables prefetch

Why this talk
• Documentation recommends higher values for storages with
high parallelism capabilities
• But there are evidences, that sometimes disabling
• Linux is a most common OS for databases
effective io concurrency leads to better results
• DBAs often run into IO problems
(https://www.postgresql.org/message-id/flat/6c7a45df-f6ab-
f2ce-6f84-9555864f6c86
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Must have optimizer settings

Why this talk

• effective cache size = 2 * shared buffers or less
• default statistics target = 100
• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Autovacuum

• autovacuum vacuum threshold = 50

• autovacuum vacuum scale factor = 0.05
Why this talk

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Autovacuum

• autovacuum vacuum threshold = 50

• autovacuum vacuum scale factor = 0.05
Why this
• autovacuum naptime = 1stalk

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Autovacuum

• autovacuum vacuum threshold = 50

• autovacuum vacuum scale factor = 0.05
Why this
• autovacuum naptime = 1stalk
• autovacuum max workers = 10

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Autovacuum

• autovacuum vacuum threshold = 50

• autovacuum vacuum scale factor = 0.05
Why this
• autovacuum naptime = 1stalk
• autovacuum max workers = 10
• autovacuum analyze threshold = 50
• Linux is a most common OS for databases
• autovacuum analyze scale factor = 0.05
• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Autovacuum

• autovacuum vacuum threshold = 50

• autovacuum vacuum scale factor = 0.05
Why this
• autovacuum naptime = 1stalk
• autovacuum max workers = 10
• autovacuum analyze threshold = 50
• Linux is a most common OS for databases
• autovacuum analyze scale factor = 0.05
• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Autovacuum

• autovacuum vacuum threshold = 50

• autovacuum vacuum scale factor = 0.05
Why this
• autovacuum naptime = 1stalk
• autovacuum max workers = 10
• autovacuum analyze threshold = 50
• Linux is a most common OS for databases
• autovacuum analyze scale factor = 0.05
• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Autovacuum

• autovacuum vacuum threshold = 50

• autovacuum vacuum scale factor = 0.05
Why this
• autovacuum naptime = 1stalk
• autovacuum max workers = 10
• autovacuum analyze threshold = 50
• Linux is a most common OS for databases
• autovacuum analyze scale factor = 0.05
• DBAs often run into IO problems
• autovacuum freeze min age = 20000000 # 9.6 and older -
Most of
default is most •likely the information
enough, olderon versions
topic is written by kerneldevelopers
often require up
(for kernel developers) or is checklist-style
to 1B
• Checklists are useful, but up to certain workload
• autovacuum freeze table age = 15000000

dataegret.com
dataegret.com
 Logging

log_directory = ’’/var/log/postgresql’’
Why this talk
log_filename = ’’postgresql-%Y-%m-%d.log’’
log_rotation_age = 1d
log_rotation_size = 0
log_min_error_statement = error
log_min_duration_statement = 1000
log_checkpoints = on
log_line_prefix = ’’%m %p %u@%d from %h [vxid:%v txid:%x] [%i] ’’
log_lock_waits = on • Linux is a most common OS for databases
log_statement = ’’none’’
log_replication_commands = on
log_temp_files = 0 • DBAs often run into IO problems
log_timezone = ’’Europe/Berlin’’
• Most of the information on topic is written by kerneldevelopers
(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Don’t forget about one very useful extension

Why this talk

• shared preload libraries = ’pg stat statements’
• pg stat statements.max = 10000
• pg stat statements.track = common
• Linux is a most top OS for databases
• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com
 Questions?

Why this talk

[email protected]

• Linux is a most common OS for databases

• DBAs often run into IO problems

• Most of the information on topic is written by kerneldevelopers

(for kernel developers) or is checklist-style

• Checklists are useful, but up to certain workload

dataegret.com
dataegret.com