1.
Phần mềm cài đặt
· Oracle GoldenGate for BigData
· Oracle GoldenGate for Oracle
· JDK 1.8
· Kafka 2.7.0 ([Link]
2. Cấu hình GoldenGate trên database nguồn
2.1. Install Oracle GoldenGate for Oracle
[oracle@oracle-sonct2 ~]$ cd /u01/soft
[oracle@oracle-sonct2 soft]$ unzip [Link]
[oracle@oracle-sonct2 ~]$ cd /u01/soft/fbo_ggs_Linux_x64_shiphome/Disk1/
[oracle@oracle-sonct2 Disk1]$ ./runInstaller
2.2. Cấu hình manager
GGSCI (oracle-sonct2) 2> edit param mgr
PORT 7809
LAGREPORTMINUTES 60
LAGCRITICALMINUTES 20
GGSCI (oracle-sonct2) 3> start mgr
GGSCI (oracle-sonct2) 4> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
2.3. Cấu hình database nguồn
SQL> select
NAME,SUPPLEMENTAL_LOG_DATA_MIN,SUPPLEMENTAL_LOG_DATA_PK,SUPPLEMENTAL
_LOG_DATA_UI,SUPPLEMENTAL_LOG_DATA_FK,SUPPLEMENTAL_LOG_DATA_ALL from
v$database;
NAME SUPPLEME SUP SUP SUP SUP
--------- -------- --- --- --- ---
TESTDB NO NO NO NO NO
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
SQL> alter database add SUPPLEMENTAL LOG DATA (primary key) columns;
SQL> alter database add SUPPLEMENTAL LOG DATA (Foreign key) columns;
SQL> alter database add SUPPLEMENTAL LOG DATA (all) columns;
SQL> alter database add SUPPLEMENTAL LOG DATA (unique index) columns;
SQL> select
NAME,SUPPLEMENTAL_LOG_DATA_MIN,SUPPLEMENTAL_LOG_DATA_PK,SUPPLEMENTAL
_LOG_DATA_UI,SUPPLEMENTAL_LOG_DATA_FK,SUPPLEMENTAL_LOG_DATA_ALL from
v$database;
NAME SUPPLEME SUP SUP SUP SUP
--------- -------- --- --- --- ---
TESTDB YES YES YES YES YES
SQL> select NAME,LOG_MODE,FORCE_LOGGING from v$database;
NAME LOG_MODE FORCE_LOGGING
--------- ------------ ---------------------------------------
TESTDB NOARCHIVELOG NO
SQL> show parameter recover;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest string
db_recovery_file_dest_size big integer 0
SQL> alter system set db_recovery_file_dest_size=10G scope=both;
SQL> alter system set db_recovery_file_dest='/u01/app/oracle/fast_recovery_area'
scope=both;
SQL> shutdown immediate
SQL> startup mount
SQL> alter database archivelog;
SQL> alter database force logging;
SQL> alter database open;
SQL> select NAME,LOG_MODE,FORCE_LOGGING from v$database;
NAME LOG_MODE FORCE_LOGGING
--------- ------------ ---------------------------------------
TESTDB ARCHIVELOG YES
SQL> alter system set enable_goldengate_replication=true scope=both;
2.4. Cấu hình user ggadmin
SQL> create user c##GGADMIN identified by xxx;
SQL> grant dba to c##ggadmin container=all;
SQL> exec DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE('C##GGADMIN',
container => 'all');
2.5. Cấu hình extract
GGSCI (staging-02) 1> GGSCI (oracle-sonct2) 3> dblogin userid c##ggadmin@pdbtest,
password xxx
GGSCI (oracle-sonct2 as ggadmin@testdb/PDBTEST) 5> add trandata testuser.test_table
GGSCI (oracle-sonct2) 2> dblogin userid c##ggadmin, password xxx
GGSCI (oracle-sonct2 as c##ggadmin@testdb/CDB$ROOT) 4> REGISTER EXTRACT ex1,
DATABASE CONTAINER (pdbtest)
GGSCI (oracle-sonct2 as ggadmin@testdb/CDB$ROOT) 6> edit param ex1
EXTRACT ex1
USERID ggadmin@PDBTEST, PASSWORD xxx
DBOPTIONS ALLOWUNUSEDCOLUMN
EXTTRAIL ./dirdat/ea
TABLE TESTUSER.TEST_TABLE;
GGSCI (oracle-sonct2 as c##ggadmin@testdb/CDB$ROOT) 5> ADD EXTRACT ex1,
INTEGRATED TRANLOG, BEGIN NOW
GGSCI (oracle-sonct2 as c##ggadmin@testdb/CDB$ROOT) 16> ADD EXTTRAIL
./dirdat/ea, EXTRACT ex1, MEGABYTES 1024
GGSCI (oracle-sonct2 as c##ggadmin@testdb/CDB$ROOT) 18> start ex1
GGSCI (oracle-sonct2 as c##ggadmin@testdb/CDB$ROOT) 19> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EX1 [Link] [Link]
3. Cấu hình GoldenGate for Kafka
3.1. Install Oracle GoldenGate for bigdata
[oracle@oracle-sonct2 ~]$ cd /u01/soft
[oracle@oracle-sonct2 soft]$ unzip [Link]
[oracle@oracle-sonct2 soft]$ tar -xvf OGG_BigData_Linux_x64_19.[Link].tar -C
/u01/app/ogg/kafka/
3.2. Install JDK 1.8
[oracle@oracle-sonct2 app]$ cd /u01/soft
[oracle@oracle-sonct2 soft]$ tar -zxvf [Link] -C /u01/app/ogg/jdk/
Thêm biến môi trường vào .bash_profile
export JAVA_HOME=/u01/app/ogg/jdk/jdk1.8.0_271
export PATH=$JAVA_HOME/bin:$PATH
export
LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64:$JAVA_HOME/jre/lib/amd64/server:
$LD_LIBRARY_PATH
3.3. Install Kafka
[oracle@oracle-sonct2 ~]$ cd /u01/soft
[oracle@oracle-sonct2 soft]$ tar -xzvf kafka_2.[Link] -C /u01/app/ogg
Thêm biến môi trường vào .bash_profile
export KAFKA_HOME=/u01/app/ogg/kafka_2.13-2.7.0
export PATH=$KAFKA_HOME/bin:$JAVA_HOME/bin:$PATH
3.4. Khởi tạo subdirs cho goldengate
[oracle@oracle-sonct2 ~]$ cd /u01/app/ogg/kafka/
[oracle@oracle-sonct2 kafka]$ ./ggsci
GGSCI (oracle-sonct2) 1> create subdirs
Creating subdirectories under current directory /u01/app/ogg/kafka
Parameter file /u01/app/ogg/kafka/dirprm: created.
Report file /u01/app/ogg/kafka/dirrpt: created.
Checkpoint file /u01/app/ogg/kafka/dirchk: created.
Process status files /u01/app/ogg/kafka/dirpcs: created.
SQL script files /u01/app/ogg/kafka/dirsql: created.
Database definitions files /u01/app/ogg/kafka/dirdef: created.
Extract data files /u01/app/ogg/kafka/dirdat: created.
Temporary files /u01/app/ogg/kafka/dirtmp: created.
Credential store files /u01/app/ogg/kafka/dircrd: created.
Masterkey wallet files /u01/app/ogg/kafka/dirwlt: created.
Dump files /u01/app/ogg/kafka/dirdmp: created.
3.5. Cấu hình manager
GGSCI (oracle-sonct2) 2> edit param mgr
PORT 7810
LAGREPORTMINUTES 60
LAGCRITICALMINUTES 20
GGSCI (oracle-sonct2) 4> start mgr
Manager started.
GGSCI (oracle-sonct2) 6> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
3.6. Cấu hình kết nối kafka
[oracle@oracle-sonct2 ~]$ cd /u01/app/ogg/kafka/dirprm/
[oracle@oracle-sonct2 dirprm]$ vi [Link]
[Link]
28 Dec 2020, 05:59 PM
[oracle@oracle-sonct2 dirprm]$ vi custom_kafka_producer.properties
custom_kafka_producer.[Link]
28 Dec 2020, 05:59 PM
3.7. Cấu hình replicat
GGSCI (oracle-sonct2) 1> edit param RX1
REPLICAT RX1
TARGETDB LIBFILE [Link] SET property=dirprm/[Link]
GROUPTRANSOPS 10000
MAP [Link].TEST_TABLE, TARGET TESTUSER.TEST_TABLE;
GGSCI (oracle-sonct2) 3> add replicat RX1, exttrail /u01/app/ogg/oracle/dirdat/ea
GGSCI (oracle-sonct2) 5> start RX1
GGSCI (oracle-sonct2) 6> info all
