Add GPDB 6 support #1833
Add GPDB 6 support #1833
Conversation
andr-sokolov
force-pushed
the
ADBDEV-2911
branch
2 times, most recently
from
ea19290 to
63ec064
Compare
Add fork version property to the src/build/postgres/postgres.yaml file format. The default value of the property is PostgreSQL. Add the fork field to PgInterface structure. Implement the search in the pgInterface array not only by version, but also by the fork field, the value for comparison with which will be taken from the --fork command line option or from fork property of stanza or from the global section. For backward compatibility purposes, the PostgreSQL interface will be searched without fork option. Add tablespaceId, walSegmentSizeDefault and pageSizeDefault functions to interface, because tablespace id format, default WAL segment size and default page size are different for PostgreSQL and GPDB. Using different page sizes do not lead to performance degradation, because vectorization is preserved.
andr-sokolov
force-pushed
the
ADBDEV-2911
branch
from
63ec064 to
de11610
Compare
|
Thank you for submitting this. It is interesting to see what it takes to get pgBackRest working with Greenplum. The bulk of the patch seems to be dealing with custom page sizes and calculating checksums for them. We are open to that part because Postgres can be configured at compile-time with different pages sizes and we've have a few requests for this support in the past. We're less interested in supporting the custom tablespace naming or pg_control for Greenplum. The main reason for this is that we expect this code to covered by integration tests (missing from this patch) which would need to be maintained going forward. Currently adding those these tests (same version, different vendor) would require changes to the Perl test code which we would rather not do while this code is being migrated to C. Also, we are not sure that we want to branch out from our core focus on standard Postgres. We think the best way forward would be to break out the variable page size code into a separate commit and base that on the value read from pg_control. Testing will still be an issue but we could lock the core code down to 8k pages until we are ready to build custom versions of Postgres for integration testing. That would be easy for you to disable when building for Greenplum. |
|
@dwsteele , Did I understand correctly that if I prepare a PR that will implement support for all page sizes used in PostgreSQL (1, 2, 4, 8, 16 and 32kb), then are you ready to merge it? |
|
Hey @andr-sokolov ! I see you put a lot of effort in this completely new area of backups. Yes, GP is different from PG. Let's work together on GP support in WAL-G. We totally understand and share needs of ADB, so we can cooperate. |
Certainly ready to review it. But yes, this is a feature we'd be happy to have in pgBackRest. |
Thanks. I'll do it later. |
|
I've been thinking about this some more, and it may be possible to add greenplum support (as well as other forks) by adding options to give pgbackrest the information about pg_control that it requires. For example, But, still makes sense to split out the page checksum patch first and then see what is left. |
|
I can't really help with the coding part but I'd be happy to help test once there is a solution available for GPDB. Any idea of a timeline? |
|
@inviscid, You can get source code to compile and test with |
|
@andr-sokolov, hello! Sorry to bother you. I decided to see how pgbackrest works with GP. Everything seems to be gone, but there is a problem, I have empty fields in these requests: Their call in the db.c file. I am currently hardcoding these values. In this case, the SHOW command in psql displays valid values. Have you encountered this problem, if so, could you write how you solved it? |
@Alexklkv123 hello! You should update GP. This commit fixes the problem https://github.com/greenplum-db/gpdb/commit/4221bffbc34c97e965e4fe3952292383c84687b7 |
@andr-sokolov hello! Thank you very much, after the update it worked! |
|
How should I configure config(pgbackrest.conf) When I run the following command The command throws the following exception Should I upgrade GPDB7( PostgreSQL 11.9) to use pgbackrest |
|
#cat /var/log/pgbackrest/pt-stanza-create.log -------------------PROCESS START------------------- 2023-03-24 14:47:33.328 P00 INFO: stanza-create command end: aborted with exception [056] |
|
select (select setting from pg_catalog.pg_settings where name = 'server_version_num')::int4, (select setting from pg_catalog.pg_settings where name = 'data_directory')::text, (select setting from pg_catalog.pg_settings where name = 'archive_mode')::text, (select setting from pg_catalog.pg_settings where name = 'archive_command')::text, (select setting from pg_catalog.pg_settings where name = 'checkpoint_timeout')::int4 My GPDB6 is a cluster environment built by three devices There was no error reported when I ran the sql in navicat |
|
Hello! Sorry for not replying sooner, I took the day off on Friday. I see
issues with *pg1-user*. Yours is *postgres*, although it should be *gpadmin*.
Perhaps there will be errors due to the fact that not all flags are created.
Attached my pgbackrest.conf.
Example commands:
*pgbackrest --stanza=master --log-level-console=info stanza-create*
*pgbackrest --stanza=master --log-level-console=info check*
*pgbackrest --stanza=master --log-level-console=info backup*
Write if you have any more questions or problems!
…On Fri, Mar 24, 2023 at 9:55 AM QL ***@***.***> wrote:
[image: 图片]
<https://user-images.githubusercontent.com/51136763/227447215-2c24f4f5-8a53-4bd7-a666-5a92be6d3f57.png>
select (select setting from pg_catalog.pg_settings where name =
'server_version_num')::int4, (select setting from pg_catalog.pg_settings
where name = 'data_directory')::text, (select setting from
pg_catalog.pg_settings where name = 'archive_mode')::text, (select setting
from pg_catalog.pg_settings where name = 'archive_command')::text, (select
setting from pg_catalog.pg_settings where name = 'checkpoint_timeout')::int4
My GPDB6 is a cluster environment built by three devices
There was no error reported when I ran the sql in navicat
—
Reply to this email directly, view it on GitHub
<#1833 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A3Q3DUN7U5T6X6GI6ZWKSWLW5VAM3ANCNFSM557YMG7Q>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Thank you for your reply. I still have some issues to resolve. Could you provide me with your email? |
|
Hi there. I'm trying to test pgbackrest with GPDB. I was able to make a backup as well as run archiving on active master. But it's unclear to me how to restore a database to the GPDB cluster while it has multiple segments. Maybe someone can describe how to restore backups made by pgbackrest to GPDB clusters? Is there any related documentation? |
@gesundes sorry for the late response. In English - https://github.com/arenadata/pgbackrest/blob/2.50-ci/README_GPDB_en.md and the same text in Russian - https://github.com/arenadata/pgbackrest/blob/2.50-ci/README_GPDB_ru.md |
|
I think we have committed everything here that is not GreenPlum specific in other PRs. Anything left that I'm not seeing? |
@dwsteele , Yes, everything from this PR has been committed |
|
OK, I'll close it then. Thank you for the patches! |
Add fork version property to the src/build/postgres/postgres.yaml file format. The default value of the property is PostgreSQL. Add the fork field to PgInterface structure. Implement the search in the pgInterface array not only by version, but also by the fork field, the value for comparison with which will be taken from the --fork command line option or from fork property of stanza or from the global section. For backward compatibility purposes, the PostgreSQL interface will be searched without fork option. Add tablespaceId, walSegmentSizeDefault and pageSizeDefault functions to interface, because tablespace id format, default WAL segment size and default page size are different for PostgreSQL and GPDB. Using different page sizes do not lead to performance degradation, because vectorization is preserved.
#1801