What happened?
There seems to be a leak in decimal (numeric) conversion. Memory is allocated in MessageContext, so it is released after BCP import finishes. Still can be a problem when importing large files.
Using the following table with a decimal column:
create table tab1 (col1 decimal(18,5))
insert into tab1 values ('41.42')
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
insert into tab1 select col1 from tab1
select count(*) from tab1
> 1048576Export it with BCP:
bcp tab1 out tab1.bcp -n -S 192.168.122.1,1433 -U jdbc_user -P 12345678
...
1048576 rows copied.
Network packet size (bytes): 4096
Clock Time (ms.) Total : 1453 Average : (721662.75 rows per sec.)
Clear the table:
Run server under mprof:
mprof run -M ./bin/postgres -D /home/alex/projects/postgres/dist/data
Import exported data back with BCP:
bcp tab1 in tab1.bcp -n -S 192.168.122.1,1433 -U jdbc_user -P 12345678
...
1048576 rows copied.
Network packet size (bytes): 4096
Clock Time (ms.) Total : 1578 Average : (664496.81 rows per sec.)
BCP import backend process shows excessive memory usage:
Will submit a patch to TdsTypeNumericToDatum shortly.
Version
BABEL_3_X_DEV (Default)
Extension
babelfishpg_tds
Which flavor of Linux are you using when you see the bug?
Fedora
Relevant log output
No response
Code of Conduct
What happened?
There seems to be a leak in decimal (numeric) conversion. Memory is allocated in
MessageContext, so it is released after BCP import finishes. Still can be a problem when importing large files.Using the following table with a decimal column:
Export it with BCP:
Clear the table:
Run server under mprof:
Import exported data back with BCP:
BCP import backend process shows excessive memory usage:
Will submit a patch to
TdsTypeNumericToDatumshortly.Version
BABEL_3_X_DEV (Default)
Extension
babelfishpg_tds
Which flavor of Linux are you using when you see the bug?
Fedora
Relevant log output
No response
Code of Conduct