Rebuild MySQL Master Master Replication after ERROR: Got fatal error 1236 from master when reading data from binary log

Rebuild MySQL Master Master Replication after receiving the error below. This procedure can also be used to setup Mysql Master / Master replication.

Possible error in the mysql error log:

Got fatal error 1236 from master when reading data from binary log: 
'Found old binary log without GTIDs while looking for the oldest 
binary log that contains any GTID that is not in the given 
gtid set', Error_code: 1236

This post is about the following configuration. A pair of MySQL servers running CENTOS 6.5 and MySQL 5.6.
The MySQL servers are running salves to one another Multi-Master. The SQL clients write to a Virtual IP which is configured to float between the MASTER and SLAVE in the event of a failover.
Reference Architecture:
MYSQLSERVER01 – Primary Master
MYSQLSERVER02 – Backup Master
Keepalived – Virtual IP

STEP 1: ON MYSQLSERVER02
Log into MySQL Workbench and execute the following commands

STOP SLAVE;
RESET SLAVE;
RESET MASTER;

STEP 2: ON MYSQLSERVER01
Log into MySQL Workbench and execute the following commands

STOP SLAVE;
RESET SLAVE;
RESET MASTER;

STEP 3: ON MYSQLSERVER01 take a backup with mysqldump from the master:
SSH into the server and execute the following

mysqldump --all-databases --single-transaction --triggers --routines --events --user=root -p > /tmp/dump.sql

STEP 4: ON MYSQLSERVER01 transfer the mysqldump backup file form MYSQLSERVER01 to MYSQLSERVER02
SSH into the server and execute the following

scp /tmp/dump.sql root@MYSQLSERVER02:/tmp/dump.sql

STEP 5: ON MYSQLSERVER01 load the mysqldump file
SSH into the server and execute the following

mysql -u root -p  

STEP 6: ON MYSQLSERVER02

CHANGE MASTER TO MASTER_HOST='MYSQLSERVER01.lab.net', MASTER_USER='repl', MASTER_PASSWORD='replPassword', MASTER_AUTO_POSITION = 1;

STEP 7. ON MYSQLSERVER02
Verify the Slave_IO_Running and Slave_SQL_Running both have a status of YES

SHOW SLAVE STATUS;
start slave;
SHOW SLAVE STATUS;

STEP 8. ON MYSQLSERVER01
Log into MySQL Workbench and execute the following commands

CHANGE MASTER TO MASTER_HOST='MYSQLSERVER02.lab.net', MASTER_USER='repl', MASTER_PASSWORD='replPassword', MASTER_AUTO_POSITION = 1;

STEP 9. ON MYSQLSERVER01 start the slave and check the status of the slave operations
Log into MySQL Workbench and execute the following commands
Verify the Slave_IO_Running and Slave_SQL_Running both have a status of YES

SHOW SLAVE STATUS;
start slave;
SHOW SLAVE STATUS;

STEP 10. Additional testing can be done to confirm that the bi directional replication is working by inserting a record into the testdb.
Log into MySQL Workbench and execute the following commands
Perform insert test on MYSQLSERVER01, verify that the record has replicated to MYSQLSERVER02.

Configuring tsung load testing software

This document was written with the assumption that CentOS 6.5 will be used.

Host systems involved in this test.
tsung client system: client01.lab.net
web server being load tested: web01.lab.net

STEP 1. Add he following repositories to client01.lab.net, these are the Extra Packages for Enterprise Linux 6 repositories.

vim /etc/yum.repos.d/epel.repo
[epel]
name=Extra Packages for Enterprise Linux 6 - $basearch
#baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch
mirrorlist=http://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

[epel-debuginfo]
name=Extra Packages for Enterprise Linux 6 - $basearch - Debug
#baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch/debug
mirrorlist=http://mirrors.fedoraproject.org/metalink?repo=epel-debug-6&arch=$basearch
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
gpgcheck=1

[epel-source]
name=Extra Packages for Enterprise Linux 6 - $basearch - Source
#baseurl=http://download.fedoraproject.org/pub/epel/6/SRPMS
mirrorlist=http://mirrors.fedoraproject.org/metalink?repo=epel-source-6&arch=$basearch
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
gpgcheck=1

STEP 2. Install tsung load testing software

yum install tsung

STEP 3. Create a .tsung directory under the user’s home folder which you will launch tsung from.

su testuser
vim /home/testuser/.tsung/tsung.xml

STEP 4. Create the following XML file, by entering the xml below into the tsung.xml file
Update the system names in the SERVER and CLIENT tags. Also update the URL in the REQUEST tag to match your environment. Using this configuration I was able to fully utilize four IIS web servers in a load test.

<?xml version="1.0"?>
<!DOCTYPE tsung SYSTEM "/usr/share/tsung/tsung-1.0.dtd">
<tsung loglevel="notice" version="1.0">

<clients>
        <client host="client01.lab.net" weight="1" cpu="10" maxusers="40000">
                <ip value="172.16.1.20"/>
        </client>
</clients>

<servers>
      <server host="wauweb01.lab.net" port="443" type="ssl"/>
</servers>

  <load>
     <arrivalphase phase="1" duration="10" unit="minute">
        <users maxnumber="15000" arrivalrate="8" unit="second"/>
          </arrivalphase>

   <arrivalphase phase="2" duration="10" unit="minute">
      <users maxnumber="15000" arrivalrate="8" unit="second"/>
        </arrivalphase>

   <arrivalphase phase="3" duration="30" unit="minute">
      <users maxnumber="20000" arrivalrate="3" unit="second"/>
        </arrivalphase>

 </load>

 <sessions>
    <session probability="100" name="ab" type="ts_http">
           <for from="1" to="10000000" var="i">
                       <request> <http url="/webservice/test.svc?wsdl" method="GET" version="1.1"/> 
                       </request>
           </for>
     </session>
   </sessions>
</tsung>

STEP 5. On the client system start tsung, under the user which you configured the tsung.xml file

tsung start

Logging syslog messages to mysql using syslog-ng

The guide was built using CentOS 6.5, MySQL 5.6 Community Edition and Syslog-Ng. This guide assumes you already have MySQL up and running.

STEP 1. First we must download and install Extra Packages for Enterprise Linux (epel) repository for CentOS 6.

wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
sudo rpm -Uvh epel-release-6*.rpm

STEP 2. Install the following packages required for syslog-ng and mysql log destination.

yum install syslog-ng
yum install syslog-ng-libdbi
yum install libdbi-drivers
yum install libdbi-devel
yum install libdbi-dbd-mysql
yum install syslog-ng-libdbi

STEP 3. Create the following table in the syslog database in MySQL.

Create Table: CREATE TABLE `logs` (
  `host` varchar(32) DEFAULT NULL,
  `facility` varchar(10) DEFAULT NULL,
  `priority` varchar(10) DEFAULT NULL,
  `level` varchar(10) DEFAULT NULL,
  `tag` varchar(10) DEFAULT NULL,
  `datetime` datetime DEFAULT NULL,
  `program` varchar(15) DEFAULT NULL,
  `msg` text,
  `seq` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (`seq`),
  KEY `date_host` (`datetime`,`host`)
) ENGINE=InnoDB AUTO_INCREMENT=14120 DEFAULT CHARSET=utf8

STEP 4. Create a MySQL user which has DELETE, INSERT, SELECT, UPDATE on the syslog database / schema.
Create user and set password
MySQL user1
Give the user “syslog” access to write to the database.
MySQL user2

STEP 5. Begin editing the syslog-ng.conf file

vim /etc/syslog-ng/syslog-ng.conf

Sample syslog-ng configuration file to save logs to a MySQL destination.

options {
flush_lines (0);
time_reopen (10);
log_fifo_size (1000);
long_hostnames (off);
use_dns (yes);
use_fqdn (yes);
create_dirs (yes);
keep_hostname (yes);
};

# syslog-ng log source
source s_net { udp(ip("172.16.1.20") port(514)); };
perm(0644) dir_perm(0700) create_dirs(yes)); };


source s_sys {
file ("/proc/kmsg" program_override("kernel: "));
unix-stream ("/dev/log");
internal();
};

destination d_file { file("/var/log/messagestest"); };
perm(0644) dir_perm(0700) create_dirs(yes)); };
destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };

# MySQL define destination
destination d_mysql {
sql(
type(mysql)
username("syslog")
password("Pass123!")
database("syslog")
host("172.16.1.20")
table("logs")
columns("host", "facility", "priority", "level", "tag", "datetime", "program", "msg")
values("$HOST", "$FACILITY", "$PRIORITY", "$LEVEL", "$TAG","$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC","$PROGRAM", "$MSG")
indexes("datetime", "host")
);
};


filter f_kernel { facility(kern); };
filter f_default { level(info..emerg) and
not (facility(mail)
or facility(authpriv)
or facility(cron)); };
filter f_auth { facility(authpriv); };
filter f_mail { facility(mail); };
filter f_emergency { level(emerg); };
filter f_news { facility(uucp) or
(facility(news)
and level(crit..emerg)); };
filter f_boot { facility(local7); };
filter f_cron { facility(cron); };

# MySQL log to destination
log {source(s_net); destination(d_file); destination(d_mysql);};

# map source to destination fields.
log { source(s_sys); filter(f_kernel); destination(d_kern); };
log { source(s_sys); filter(f_default); destination(d_mesg); };
log { source(s_sys); filter(f_auth); destination(d_auth); };
log { source(s_sys); filter(f_mail); destination(d_mail); };
log { source(s_sys); filter(f_emergency); destination(d_mlal); };
log { source(s_sys); filter(f_news); destination(d_spol); };
log { source(s_sys); filter(f_boot); destination(d_boot); };
log { source(s_sys); filter(f_cron); destination(d_cron); };

STEP 6. Restart syslog-ng

service syslog-ng start

STEP 7. Log into MySQL and verify that logs are being saved to MySQL.

SELECT * 
FROM syslog.logs
order by datetime desc;

After completing these steps you should see the logs being saved to MySQL.
mysql-syslog-ng

How to setup Active Directory Authentication in MySQL running on Linux

This document was written using Microsoft Windows Active Directory 2012, Mysql 5.6 and CentOS 6.5. MySQL is running on CentOS 6.5. This document describes how to map an Active Directory Group to a MySQL User and authenticate against Active Directory. The completion of these steps allows Active Directory groups to be utilized within MySQL.

In this demonstration the following groups will be referenced.

mySQL_AD – This is the MySQL user which will be mapped/associated to the Active Directory Group.

AD_mysql_users – This is the Active Directory user which will be mapped to mySQL_AD.

STEP 1. Prior to getting started you need the following pam module installed.

/usr/lib64/mysql/plugin/authentication_pam.so

STEP 2. After installing the module update the my.cnf file, my.cnf is normally located in /etc/my.cnf

[mysqld]
plugin-load=authentication_pam.so

STEP 3. Restart mysql and verify that the PAM modules is installed.

service mysql restart
mysql -u root
select * from information_schema.PLUGINS\G

The following output should be returned from the PLUGIN query

*************************** 47. row ***************************
           PLUGIN_NAME: authentication_pam
        PLUGIN_VERSION: 1.0
         PLUGIN_STATUS: ACTIVE
           PLUGIN_TYPE: AUTHENTICATION
   PLUGIN_TYPE_VERSION: 1.0
        PLUGIN_LIBRARY: authentication_pam.so
PLUGIN_LIBRARY_VERSION: 1.4
         PLUGIN_AUTHOR: Georgi Kodinov
    PLUGIN_DESCRIPTION: PAM authentication plugin
        PLUGIN_LICENSE: PROPRIETARY
           LOAD_OPTION: ON
47 rows in set (0.00 sec)

STEP 4.
Create the following file: vim /etc/pam.d/mysql with the content below. This assumes you already have the MySQL Linux system joined to the Microsoft Active Directory Domain. The pam_winbind.so is a PAM module that can authenticate users against the local domain by talking to the Winbind daemon.

auth   required    pam_winbind.so
account required    pam_winbind.so

STEP 5. Create a MySQL user with a password. The password is not important because it will never be used.
mySQL_AD is the MySQL user which will be mapped to the Active Directory group.

CREATE USER 'mySQL_AD'@'%' IDENTIFIED BY 'SomePassw0rd';

STEP 6. Run the CREATE USER statement to add the MySQL user to Active Directory mapping. In this example the Active Directory group AD_mysql_users is being mapped to the MySQL user: mySQL_AD
The “mysql” in front of “AD_mysql_users=mySQL_AD” of should match the name of pam.d file e.g. (/etc/pamd./mysql)
AD_mysql_users=mySQL_AD

CREATE USER ''@''
IDENTIFIED WITH authentication_pam
AS 'mysql, AD_mysql_users=mySQL_AD';

STEP 7. Grant mySQL_AD PROXY access. This gives users in Active Directory the ability to impersonate the user: mySQL_AD. To run this command you must be logged in as MySQL root: (e.g. mysql -u root -p). Even if you are logged in with DBA access this step will not work, must be logged in under MySQL root user. I tired logging into MySQL with a user which had equivalent privileges to MySQL root and the GRANT failed. Only when I logged in with MySQL root did the GRANT succeed.

mysql -u root
GRANT PROXY ON 'mySQL_AD'@'%' TO ''@'';

STEP 8.
Do a SELECT on the table and see which MySQL users are mapped to Active Directory Groups

SELECT * FROM mysql.user;

Fig 1.
MySQL Workbench_2015-03-17_19-46-36

STEP 9.
Set the appropriate user access on the MySQL server for the MySQL user mySQL_AD. For example, add the appropriate permissions for the user to do a select on a MySQL database. Remember that mySQL_AD is the MySQL user which we created earlier and is mapped to the Active Directory Group AD_mysql_users.

STEP 10. Flush security privileges, this causes the user settings to become active

flush privileges;

You can then perform a test connection.
You may receive the message below when performing a test connection.

[root@MYSQL01 log]# mysql -u testuser -p
Enter password:
ERROR 2059 (HY000): Authentication plugin 'mysql_clear_password' cannot be loaded: plugin not enabled

Set the flag to allow cleartext.

[root@MYSQL01 log]# export LIBMYSQL_ENABLE_CLEARTEXT_PLUGIN=1

If you are connecting from MySQL workbench edit your connection, select “Advanced” then “Enable Clear text Authentication Plugin”. Log into MySQL using your Active Directory Username / Password. You can verify which AD user and MySQL user you are logged in as. In Fig 2 I am authenticated as user testuser mapped to the MySQL user mySQL_AD. Note if you enable clear text authentication plugin it is recommended that you connect to MySQL using SSL via workbench.

SELECT USER(),CURRENT_USER();

Fig 2
MySQL Workbench_2015-03-18_19-57-35

The configuration below were used in support of the above configurations.

Additional configuration information in /etc/samba/smb.conf

[global]

        workgroup = LAB
        server string = Samba Server Version %v
        realm   =       LAB.NET
        netbios name = MYSQL01
        encrypt passwords = yes
        password server = dc01.lab.net
        #idmap config * : backend = rid
        #idmap config * : range = 10000-20000
        idmap config * : range = 16777216-33554431
        winbind use default domain = Yes
        winbind enum users = Yes
        winbind enum groups = Yes
        winbind nested groups = Yes
        winbind separator = +
        winbind refresh tickets = yes
        winbind offline logon = true
        template shell = /bin/bash
        template homedir = /home/%D/%U
        preferred master = no
        dns proxy = no
        wins server = dc01.lab.net
        wins proxy = no
        inherit acls = Yes
        map acl inherit = Yes
        acl group control = yes
        security = ads
        passdb backend = tdbsam

Additional configuration information in /etc/krb5.conf

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = LAB.NET
 dns_lookup_realm = true
 dns_lookup_kdc = true
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 proxiable = true

[realms]
 LAB.NET = {
  default_domain = LAB.NET
  kdc = DC01.LAB.NET
  admin_server = DC01.LAB.NET
 }

[domain_realm]
 .lab.net = LAB.NET
 lab.net = LAB.NET
 lab = LAB.NET

[appdefaults]
        pam = {
        ticket_lifetime         = 1d
        renew_lifetime          = 1d
        forwardable             = true
        proxiable               = true
        retain_after_close      = false
        minimum_uid             = 0
        debug                   = true
        }

Additional configuration information in /etc/nsswitch.conf

passwd:     files sss ldap winbind
shadow:     files sss ldap winbind
group:      files sss ldap winbind

Additional configuration information in /etc/pam.d/system-auth

auth        required      pam_env.so
auth        sufficient    pam_fprintd.so
auth        sufficient    pam_unix.so nullok try_first_pass
auth        sufficient    pam_krb5.so use_first_pass
auth        sufficient    pam_winbind.so cached_login use_first_pass
auth        requisite     pam_succeed_if.so uid &gt;= 500 quiet
auth        required      pam_deny.so

account     required      pam_unix.so
account     sufficient    pam_localuser.so
account     sufficient    pam_succeed_if.so uid &lt; 500 quiet
account     [default=bad success=ok user_unknown=ignore] pam_krb5.so
account     [default=bad success=ok user_unknown=ignore] pam_winbind.so cached_login
account     required      pam_permit.so

password    requisite     pam_cracklib.so try_first_pass retry=3 type=
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password    sufficient    pam_krb5.so use_authtok
password    sufficient    pam_winbind.so cached_login use_authtok
password    required      pam_deny.so

session     optional      pam_keyinit.so revoke
session     required      pam_limits.so
session     optional      pam_mkhomedir.so skel=/etc/skel umask=0027
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session     required      pam_unix.so
session     optional      pam_krb5.so

Reference : http://dev.mysql.com/doc/refman/5.6/en/proxy-users.html

Installing Ubuntu 14.04 on a Intel NUC

These are the instructions I used to install Ubuntu 14.04 on a Intel NUC BOXD54250WYKH1.
I was having difficulty in getting the Ubuntu installer to recognize my SATA 2.5″ Crucial_CT240M500SSD1 SSD. Every time the Ubuntu install would get to the disk partitioning screen it would crash. It turns out my partition table was formatted in MSDOS and is required to be in GPT. Some of the screen shots I generated using a Ubuntu VMware virtual machine, however the same steps apply to a physical machine. In Fig 1 you can see that the partition table type is msdos.

STEP 1. Boot to the Ubuntu Live CD

STEP 2. Verify that your partition table is not GPT, by running diskparted. Select the disk you will be installing Ubuntu on and select VIEW and Device Information.

sudo diskparted

Fig 1.
partition-table-msdos

STEP 3. Run gdisk utility

sudo gdisk /dev/sda

The gdisk utility states that it found an invalid GPT, and will convert the MBR to a GPT.
Fig 2.
gdisk

Below I create a new partition 2 that is 550MB in size and flagged as an EFI system partition.

Command (? for help): n
Partition number (2-128, default 2):
First sector 34-33554398, default = 1077248) or {+-}size{KMGTP}
Last sector (1077248-33554398, default = 3354398) or {+-}size{KGTP}: +550M
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): ef00
Change type of partition to 'EFI System'
Command (? for help): w

Fig 3.
EFI-system-partition

STEP 4. Format the EFI system partition as FAT32

sudo mkfs.msdos -F 32 /dev/sda1

Fig 4.
gpt-device

After completing these steps I was able to restart the Ubuntu installer and install Ubuntu 14.04 on my NUC and Crucial SSD.

Testing network latency using TCP instead of ICMP

This configuration was built using CentOS 6.5. Sometimes I am unable to test network latency because ICMP is blocked. Under those circumstances Hping2, is useful because it uses TCP to measure network latency. However, one capability that hping lacks is a time stamp associated with each measurement. Using BASH scripting we are able to add a time stamp.

STEP 1. Install hping2

yum install hping2

STEP 2. create script which adds a time stamp to each measurement

while true; do \
    hping2 www.google.com -S -p 443 -i 1 -c 1 | grep "len=" \
    | sed -e "s/^/`date` /g"; \
    sleep 1; \
done

Script output

--- www.google.com hping statistic ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 15.9/15.9/15.9 ms
Mon Mar  9 07:08:15 CDT 2015 len=46 ip=216.58.216.228 ttl=50 id=50666 sport=443 flags=SA seq=0 win=42900 rtt=15.9 ms

--- www.google.com hping statistic ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 18.5/18.5/18.5 ms
Mon Mar  9 07:08:16 CDT 2015 len=46 ip=216.58.216.228 ttl=50 id=1590 sport=443 flags=SA seq=0 win=42900 rtt=18.5 ms

SQL SERVER Powershell fails with “set-executionpolicy RemoteSigned -scope process -Force”

This is the second or third time I have run into the message below when trying to run a Powershell script from a SQL SERVER agent job. Even after running set-executionpolicy to something other then RemoteSigned.
This issue has occurred while running Microsoft Windows 2012 and SQL SERVER 2012. And I was running the SQL AGENT step using a proxy account.

Executed as user: DOMAIN\testuser01. A job step received an error at line 1 in a PowerShell script. 
The corresponding line is 'set-executionpolicy RemoteSigned -scope process -Force'. 
Correct the script and reschedule the job. The error information returned by PowerShell is: 'Security error.  '.  Process Exit Code -1.  
The step failed.

Step 1 create credential
Step 2 create proxy from credential
Step 3 Assign proxy account to job step
Step 4. Paste the Powershell code into the job step and run the SQL AGENT job, you receive the message above.
Step 5. In an attempt to remediate the issue I performed the following steps.
Start SQL Server Management Studio as Administrator, or you cannot change the Powershell execution policy

Right click on SQL SERVER instance from from SQL Server Management Studio > Start Powershell

PS SQLSERVER:\SQL\SQLSERVER01\INSTANCE01&gt; set-executionpolicy -scope Process unrestricted
PS SQLSERVER:\SQL\SQLSERVER01\INSTANCE01&gt; get-executionpolicy -list

                                  Scope                         ExecutionPolicy
                                  -----                         ---------------
                          MachinePolicy                            Unrestricted
                             UserPolicy                               Undefined
                                Process                            Unrestricted
                            CurrentUser                               Undefined
                           LocalMachine                            Unrestricted

Even after setting the execution policy to Unrestricted the Powershell script would still not execute. Note I do not recommend setting the execution policy to Unrestricted. This blog post is geared towards figuring out why Powershell is not honoring the set-executionpolicy cmdlet. The fix was to modify registry key below and set the registry key to the appropriate execution policy.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\ShellIds\Microsoft.SqlServer.Management.PowerShell.sqlps110\Unrestricted
REG_SZ Unrestricted

Setup Linux CIFS AutoFS (automount) using kerberos authentication

Written using CentOS 6, Windows 2012 Active Directory
This guide was written assuming you already have Kerberos authentication working. In this post I will describe how to mount a Windows CIFS share from a Linux system using Kerberos authentication to a Windows Active Directory domain. In addition, the users credentials will be stored securely in a keytab file.

Step 1. verify you can get a Kerberos ticket

kinit [email protected]
Password for [email protected]:
klist

Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [email protected]

Valid starting     Expires            Service principal
03/05/15 13:57:02  03/05/15 23:57:02  krbtgt/[email protected]
        renew until 03/12/15 14:57:02

Step 2. Run kdestory to clear the Kerberos cache

kdestory

Step 3. Create a keytab file which will be used to store your credentials in an encrypted format. Later we will use the keytab file to get your Kerberos ticket

-k specifies the keytype
-e specifies the encryption type
wkt writes the keytab file

[user1@vm01 ~]$ ktutil
    ktutil:  addent -password -p [email protected] -k 1 -e aes256-cts
    Password for [email protected]: [enter your password]
    ktutil:  wkt testuser1.keytab
    ktutil:  quit

Step 4. Edit the auto.misc file, enter the share path

vim /etc/auto.misc
share01             -fstype=cifs,rw,noperm,sec=krb5 ://fileserver/share01

Step 5. Create a crontab entry for root to refresh the Kerberos ticket every 12 hours

crontab -e

CRON will run at 1AM and 1PM to refresh the Kerberos ticket

0 1,13 * * *       /usr/bin/kinit [email protected] -k -t /root/testuser1.keytab

Step 6. Wait for the crontab to run and verify that a Kerberos ticket is received.

klist 

pfx to pem certificate conversion with openssl

I work in a mixed Linux and Windows environment. Our Certificate Authority is Windows. Unfortunately the Windows CA does not support exporting a certificate in PEM (Privacy Enhanced Mail Certificate) format. Fortunatley there is a relativity easy work around. Which requires one to download OpenSSL utilities. Most Linux applications I have supported require the certificate be in a PEM format to be readable.

In this example I export the certificate with the private key from the Windows CA. Using the openssl utility to extract the private key ( .pem file) from .pfx (Personal Information Exchange).

PFX: Defines a file format commonly used to store private with accompanying public key certificates, protected with a password-based symmetric key (standard-PKCS12).

PEM : Openssl usages PEM (Privacy Enhanced Mail Certificate) to store the private key.

If you have downloaded the openssl utility, then go to command prompt and run the following commands. If not, download it from openssl, you can either download binary or source and then compile.

Execute the following command to extract the private key from the PFX file.

STEP 1. Extract the private key from the PFX file.

openssl pkcs12 -in publicAndprivate.pfx -nocerts -out privateKey.pem

STEP 2. To extract the certificate in PEM format from the publicly signed certificate.

openssl pkcs12 -in publicAndprivate.pfx -clcerts -nokeys -out publicCert.pem

STEP 3. To remove the password from the private key file. Some applications require that the password be removed from the private key or they will fail to start.

 
openssl rsa -in privateKey.pem -out privateNoPassword.pem 

In addition, the certificate files should be secured so that only root has access to them.

MySQL and NUMA

I recently discussed NUMA in my Couchbase NUMA post. All the same principles apply with MySQL and NUMA.
NUMA imbalances in MySQL can cause extreme swapping. This configuration was performed on Mysql 5.6 community edition and CentOS 6.5.

To begin install numactl if it is already not installed on your MySQL server.

yum install numactl

Issue the command below on your physical MySQL server.

numactl --hardware

As one can see the output of the “numactl –hardware” command shows the imbalance between NUMA nodes on this particular server. Node 0 has 11,565MB free and Node 1 has 26,192MB free.

available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22
node 0 size: 32722 MB
node 0 free: 11565 MB
node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23
node 1 size: 32768 MB
node 1 free: 26192 MB
node distances:
node   0   1
  0:  10  20
  1:  20  10

The solution is to add a script to the /etc/mysql.cnf file.
This causes memory allocation to be performed in a round robin fashion across NUMA nodes.

STEP 1. Create a script file

vim /usr/sbin/mysqld_numactl

STEP 2. Add the following content to the file

# work around to startup mysqld using numactl
numactl=/usr/bin/numactl        # Adjust the path as needed
mysqld=/usr/sbin/mysqld         # Adjust the path as needed
# use exec to avoid having an extra shell around.
exec $numactl --interleave all $mysqld "$@"

STEP 3. Make script executable

chmod 750 /usr/sbin/mysqld_numactl

STEP 4. update the /etc/my.cnf

[mysqld_safe]
mysqld=mysqld_numactl

STEP 5. restart mysql to verify configuration change

service mysql restart

UPDATE:9/24/2015
With most major Linux distributions switching over to Systemd for init. I have updated this post. With Systemd in CentOS 7 it is much easier to use numactl to start up MySQL. I have provided a sample mysql.service file below.

vim /etc/systemd/system/mysql.service

Content of mysql.service

[Unit]
Description=MySQL Server
After=network.target
After=syslog.target

[Install]
WantedBy=multi-user.target

[Service]
User=mysql
Group=mysql

# Use the nice program to set the server's scheduling priority to the given value
nice=0

PIDFile=/opt/mysql/data/mysqld.pid
LimitCore=unlimited

# Execute pre and post scripts as root
PermissionsStartOnly=true

# Needed to create system tables

# Start main service
#ExecStart=
ExecStart=/usr/bin/numactl --interleave all /opt/mysql/bin/mysqld --defaults-file=/opt/mysql/my.cnf --datadir=/mnt/cp_san/mysql/data/data --socket=/tmp/mysql.sock

Environment="TZ=America/Chicago"
Environment="LD_PRELOAD=/opt/mysql/lib/libtcmalloc_minimal.so"

WorkingDirectory=/opt/mysql
# Use this to switch malloc implementation
#EnvironmentFile=-/etc/sysconfig/mysql

#Restart=on-failure
LimitNOFILE=infinity
LimitMEMLOCK=infinity

RestartPreventExitStatus=1

PrivateTmp=false

Reference: http://blog.jcole.us/2010/09/28/mysql-swap-insanity-and-the-numa-architecture/