Galera Cluster on CentOS 7

Galera Cluster on CentOS 7

Resource links

https://linuxadmin.io/galeria-cluster-configuration-centos-7/
https://mariadb.com/kb/en/library/getting-started-with-mariadb-galera-cluster/
https://mariadb.com/kb/en/library/galera-cluster-system-variables/

I will be using three servers

mysql1 = 192.168.10.160
mysql2 = 192.168.10.161
mysql3 = 192.168.10.162

With mysql1 being the master node.


Turn off firewalld

Disable selinux

reboot

Add the MariaDB repository to each of the 3 or more nodes

vi /etc/yum.repos.d/MariaDB.repo

Insert the following repository information and save the file

[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/10.1/centos7-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1

Install the packages from yum, Galera is included when these are installed when using 10.1 or greater

yum -y install MariaDB-server MariaDB-client MariaDB-common

rsync is the default to perform the replication so install that. Also lsof (list open files)

yum install -y rsync lsof

Make sure each of the MariaDB instances starts on reboot

systemctl enable mariadb

Galera Master Node Configuration

After installing MariaDB on the master node edit the server.cnf file

vi /etc/my.cnf.d/server.cnf

add the following under the [galera]

binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
bind-address=0.0.0.0
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://192.168.10.160,192.168.10.161,192.168.10.162"
#
## Galera Cluster Configuration
wsrep_cluster_name="cluster1"
## Galera Synchronization Configuration
wsrep_sst_method=rsync
## Galera Node Configuration
####unigue per node
wsrep_node_address="192.168.10.160"
####unigue per node
wsrep_node_name="mysql1"

wsrep_on=ON – Setting this to ON enables replication. In MariaDB 10.1, replication is turned off as a default, so this needs to be explicitly stated.
wsrep_cluster_address  – This is where we specify each of the IP addresses for the nodes delineated by a comma. The primary node is always the first IP address, this this case its 192.168.10.160
wsrep_cluster_name – Is the name of the cluster, you can name this anything you want
wsrep_node_address – Is the IP address of the node you are configuring
wsrep_node_name – This is the name of the node you are currently configuring, it can be named anything you want, it just needs to be unique.

Under the [mysqld] section add a log location (if you don’t, it will log to the main syslog)

log_error=/var/log/mariadb.log

Once you have finished editing and saved server.cnf, go ahead and create the error log

touch /var/log/mariadb.log

Give the error log the appropriate permissions:

chown mysql:mysql /var/log/mariadb.log

You can now start the new master node by typing the following

galera_new_cluster

After you have started it, make sure it has bound to the correct ports using lsof

Port 4567 is for replication traffic:

# lsof -i:4567
 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
 mysqld 4121 mysql 11u IPv4 34770 0t0 TCP *:tram (LISTEN)
Port 3306 is for MySQL client connections:

# lsof -i:3306
 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
 mysqld 4121 mysql 26u IPv4 34787 0t0 TCP *:mysql (LISTEN)

mysql

Then check the cluster size

MariaDB [(none)]> SHOW STATUS LIKE 'wsrep_cluster_size';
 +--------------------+-------+
 | Variable_name      | Value |
 +--------------------+-------+
 | wsrep_cluster_size | 1     |
 +--------------------+-------+
It should say 1 at this point because only the primary node is connected.

Adding Additional Nodes To Galera

After installing MariaDB on the addtional nodes, you will want to copy the [galera] section of /etc/my.cnf.d/server.cnf that We created earlier and insert it into the server.cnf on each of the additional nodes. The only lines that will each on each of the additional nodes will be the the following:

wsrep_node_address="192.168.10.161"
wsrep_node_name="mysql2"

The wsrep_node_address will be the IP address of the node you are configuring and the wsrep_node_name will be the name of that node.

After you have finished each of the servers configuration files, you can start them normally

systemctl start mariadb

As each node connects to the cluster you should see the wsrep_cluster_size increase:

MariaDB [(none)]> SHOW STATUS LIKE 'wsrep_cluster_size';
 +--------------------+-------+
 | Variable_name      | Value |
 +--------------------+-------+
 | wsrep_cluster_size | 3     |
 +--------------------+-------+

You will also see nodes join in the log:

WSREP: Member 1.0 (centos7-vm2) synced with group.
The logs will also indicate when a node as left the group:

WSREP: forgetting 96a5eca6 (tcp://192.168.1.101:4567)
WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2

You can view the full configuration of Galera by typing the following:

MariaDB [(none)]> show status like 'wsrep%';

Testing Replication On The Galera Cluster. On any node create a new db with

MariaDB [(none)]> create database testdb1;

You should see the testdb1 database appear on the other nodes as well.

MySQL allow remote access

MySQL allow remote access

Say you want to allow the user root with the password of password remote access to the database of test.

GRANT ALL PRIVILEGES ON test.* TO 'root'@'%' IDENTIFIED BY 'password';

FLUSH PRIVILEGES;
quit;

CentOS 7 install mysql connector for python3.6

CentOS 7 install mysql connector for python3.6

Go to website

https://pypi.org/project/mysqlclient/#files

and download the file

mysqlclient-1.3.12.tar.gz

place in /root dir

Now from server console as root user (or su)

cd /root

gzip -d mysqlclient-1.3.12.tar.gz

tar -xvf mysqlclient-1.3.12.tar

cd /root/mysqlclient-1.3.12

yum install -y mariadb-devel

python3.6 setup.py build

python3.6 setup.py install

Now in py script

#!/usr/bin/python3.6

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# execute SQL query using execute() method.
cursor.execute("SELECT VERSION()")

# Fetch a single row using fetchone() method.
data = cursor.fetchone()
print ("Database version : %s " % data)

# disconnect from server
db.close()

If you use this in a web browser

#!/usr/bin/python3.6
import os
import cgi
import MySQLdb

# Set content typr for web browser use
print ("Content-type: text/html\n")

# Open database connection
db = MySQLdb.connect("localhost","root","password","information_schema" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# execute SQL query using execute() method.
cursor.execute("SELECT VERSION()")

# Fetch a single row using fetchone() method.
data = cursor.fetchone()
print ("Database version : %s " % data)

# disconnect from server
db.close()

Python3.6 My personal 101 notes

Python3.6 My personal 101 notes

Miscellaneous operating system interfaces
import os
Common Gateway Interface support
import cgi

CGI Env Variables
http://www.cgi101.com/book/ch3/text.html


# Set content type for web browser use
print ("Content-type: text/html\n")

# Begin the web page html tags
print ("<html>")
print ("<body>")

# End the web page html tags
print ("</body>")
print ("</html>" )


# Get Values from a form
# So say form.html submites to /cgi-bin/form.py
form = cgi.FieldStorage()
if "name" not in form or "addr" not in form:
    print "<H1>Error</H1>"
    print "Please fill in the name and addr fields."
    return
print "<p>fname:", form["fname"].value
print "<p>lname:", form["lname"].value
...further form processing here...

# Get data from fields assign to vars and print them
# using form = cgi.FieldStorage()
first_name = form.getvalue('fname')
last_name  = form.getvalue('lname')
my_age = form.getvalue('age')
print (first_name)
print ("<br>")
print (last_name)
print ("<br>")
print (my_age)
print ("<br>")

# Print form values using form = cgi.FieldStorage()
print ("first name: " + form["fname"].value)
print ("<br>")
print ("last name: " + form["lname"].value)
print ("<br>")
print ("age: " + form["age"].value)
print ("<br>")

# Print some os enviroment vars
# using import os
ra = os.environ['REMOTE_ADDR']
print (ra)
print ("<br>")
ss = os.environ['SERVER_SOFTWARE']
print (ss)
print ("<br>")


# This is a formatted web ui output of some stuff
print ("Content-type: text/html\n")
print ("<html>")
print ("<body>")
print ("<h1>Hello " + first_name + " " + last_name + ". Your age is " + age + "</h1>")
print ("The IP address of your client is: " + ra)
print ("<br>")
print ("The software on this Python Server is: " + ss)
print ("</body>")
print ("</html>" )


# Read Write files with Python
http://www.pythonforbeginners.com/files/reading-and-writing-files-in-python

print ("<h1>Writing text file now to testfile.txt</h1>")

# Set file path and name as var of path
path = '/var/www/cgi-bin/testfile.txt'

file = open(path,'w')
file.write("Hello World\n")
file.write("This is our new text file\n")
file.write("and this is another line.\n")
file.write("Why? Because we can.\n")
file.close()

print ("<h1>Reading text file testfile.txt</h1>")
file = open(path,'r')
print (file.read())
file.close()

print ("<h1>Reading first 10 characters</h1>")
file = open(path,'r')
print (file.read(10))
file.close()

print ("<h1>Reading file by line</h1>")
file = open(path,'r')
print (file.readline())
file.close()

print ("<h1>Reading file all lines</h1>")
file = open(path,'r')
print (file.readlines())
file.close()

print ("<h1>Reading file looping over file object</h1>")
file = open(path,'r')
for line in file:
    print (line)
...
file.close()

Python3.6 Processing user Input

Jacked from http://interactivepython.org/runestone/static/webfundamentals/CGI/forms.html

Processing user Input

Lets start with a basic page with a form.

<html>
  <body>
      <form action='cgi-bin/hello2.py' method='get'>
          <label for="myname">Enter Your Name</label>
          <input id="myname" type="text" name="firstname" value="Nada" />
          <input type="submit">
      </form>
  </body>
</html> 

There are two important attributes on the form tag:

• method: this tells the browser which http method to use when submitting the form back to the server. The options are get or post.
• action: This tells the browser the URL to use when submitting the form.

The input type submit renders as a button in the form. The purpose of this input type is to cause the form to be submitted back to the web server.

#!/usr/bin/env python
import os

print "Content-type: text/html\n"

qs = os.environ['QUERY_STRING']
if 'firstname' in qs:
    name = qs.split('=')[1]
else:
    name = 'No Name Provided'

print "<html>"
print "<body>"
print "<h1>Hello %s</h1>" % name
print "</pre>"
print "</body>"
print "</html>" 

The new cgi script must now check to see if the QUERY_STRING environment variable contains the string firstname. Note that that firstname in the query string corresponds to the name attribute of the input element.

When you press the submit button for a form, the web browser iterates over all of the input elements in the form, and collects the name value attributes of each. These are put together into a string that becomes part of the URL. The name value pairs are added after the usual URL information using the form: ?firstname=Sheldon&lastname=Cooper The ? separates the query string information from the URL itself. The & separates each name value pair.

The following figure gives you a good sense for the flow of how our little application works.

Combining into One File

Lets now combine our application into a single file. Using the following flow:

1. If there is no QUERY_STRING simply return the HTML for the form.
2. If there is a QUERY_STRING then do not display the form, simply display the Hello greeting to the name stored in the QUERY STRING.

Along the way we will clean up our code and refactor it into a couple of functions.

#!/usr/bin/env python
import os

headers = ["Content-type: text/html"]
qs = os.environ['QUERY_STRING']

def sendHeaders():
    for h in headers:
        print h
    print "\n"

def sendForm():
    print '''
    <html>
      <body>
          <form action='cgi-bin/hellobetter.py' method='get'>
              <label for="myname">Enter Your Name</label>
              <input id="myname" type="text" name="firstname" value="Nada" />
              <input type="submit">
          </form>
      </body>
    </html>
    '''

def sendPage(name):
    print '''
    <html>
      <body>
        <h1>Hello {0}</h1>
      </body>
    </html>
    '''.format(name)

if not qs:
    sendHeaders()
    sendForm()
else:
    if 'firstname' in qs:
        name = qs.split('=')[1]
    else:
        name = 'No Name Provided'
    sendHeaders()
    sendPage(name) 

The headers list is to set us up with a pattern that will be useful later. Sometimes we don’t know right away what headers we may want to send. We’ll see that in the next section. So we can defer sending the headers until we have done all of our processing and are ready to send back the results. To add a header to our response we can simply append the string to the list of headers.

The other functions, sendPage and sendForm reduce the number of print statements we need by making use of Python’s triple quoted strings, and string formatting.

Next Section - Cookies

 

CentOS 7 install and load mod_wsgi in Apache for Python

CentOS 7 install and load mod_wsgi in Apache for Python

Must have httpd-develinstalled

yum install httpd-devel

Source code tar balls can be obtained from:

https://github.com/GrahamDumpleton/mod_wsgi/releases

wget the file you want

uncompress and untar

tar xvfz mod_wsgi-X.Y.tar.gz

cd into the mod_wsgi dir

Now configure, make, and make install

./configure

make

make install

Make sure the module gets loaded in Apache at restart by adding into the main Apache “httpd.conf” configuration file by adding the following

LoadModule wsgi_module modules/mod_wsgi.so

Now restart Apache

systemctl restart httpd

Now check to make sure mod is loaded

apachectl -M


Now to setup for WSGIScriptAlias. This was taken from

https://modwsgi.readthedocs.io/en/master/configuration-directives/WSGIScriptAlias.html

add the following to the httpd.conf file

WSGIScriptAlias /wsgi-scripts/ /var/www/wsgi-scripts/
<Directory /var/www/wsgi-scripts>
Order allow,deny
Allow from all
</Directory>


mkdir /var/www/wsgi-scripts/

vi /var/www/wsgi-scripts/myapp.wsgi

def application(environ, start_response):
    status = '200 OK'
    output = b'Hello World!'
    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)
    return [output]

Now chmod the file to 705

chmod 705 /var/www/wsgi-scripts/myapp.wsgi

Restart httpd with

systemctl restart httpd

Now see your app in the web browser at

http://your-ip-here/wsgi-scripts/myapp.wsgi

CentOS 7 Enable CGI executing for use of Python script

CentOS 7 Enable CGI executing for use of Python script

See the post "CentOS 7 install and setup for python3.6" at

http://glenewhittenberg.blogspot.com/2018/06/centos7-install-and-setup-for-python36_1.html

before continuing

By default, CGI is allowed under the "/var/www/cgi-bin" directory.

[root@python /]# grep "^ *ScriptAlias" /etc/httpd/conf/httpd.conf
    ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"
   
Create test page in /var/www/cgi-bin/ directory

[root@python /]# vi /var/www/cgi-bin/test.py

#!/usr/bin/python3.6
print ("Content-type: text/html\n\n")
print ("<html>\n<body>")
print ("<div style=\"width: 100%; font-size: 40px; font-weight: bold; text-align: center;\">")
print ("Python Script Test Page")
print ("</div>\n</body>\n</html>")
Now chmod so the file is executable to all (-rwx---r-x)

[root@python /]# chmod 705 /var/www/cgi-bin/test.py

Now run in web browser

http://your-ip-here/cgi-bin/test.py

CentOS 7 install and setup for python3.6

CentOS 7 install and setup for python3.6

cp /etc/sysconfig/selinux /etc/sysconfig/selinux.bak
sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/sysconfig/selinux

cp /etc/selinux/config /etc/selinux/config.bak
sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config

systemctl disable firewalld
systemctl stop firewalld

service iptables stop
service ip6tables stop
chkconfig iptables off
chkconfig ip6tables off

yum -y install bind-utils traceroute net-tools ntp* gcc glibc glibc-common gd gd-devel make net-snmp openssl-devel xinetd unzip libtool* make patch perl bison flex-devel gcc-c++ ncurses-devel flex libtermcap-devel autoconf* automake* autoconf libxml2-devel cmake sqlite* wget ntp* lm_sensors ncurses-devel qt-devel hmaccalc zlib-devel binutils-devel elfutils-libelf-devel wget bc gzip uuid* libuuid-devel jansson* libxml2* sqlite* openssl* lsof NetworkManager-tui mlocate yum-utils kernel-devel nfs-utils tcpdump git vim gdisk parted

yum -y groupinstall "Development Tools"
yum -y update
yum -y upgrade

cd /root
echo ':color desert' > .vimrc

systemctl disable kdump.service

reboot


yum -y install epel-release
yum -y install stress htop iftop iotop hddtemp smartmontools iperf3 sysstat mlocate
updatedb

---install LAMP

yum -y install httpd mariadb-server mariadb php php-mysql
systemctl enable httpd.service
systemctl start httpd.service
systemctl status httpd.service

Make sure it works with:
http://your_server_IP_address/

systemctl enable mariadb
systemctl start mariadb
systemctl status mariadb
mysql_secure_installation

vi /var/www/html/info.php
<?php phpinfo(); ?>

http://your_server_IP_address/info.php


---End install LAMP



Below was jacked from https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-centos-7

Let’s first make sure that yum is up to date by running this command:

    sudo yum -y update

The -y flag is used to alert the system that we are aware that we are making changes, preventing the terminal from prompting us to confirm.

Next, we will install yum-utils, a collection of utilities and plugins that extend and supplement yum:

    sudo yum -y install yum-utils

Finally, we’ll install the CentOS Development Tools, which are used to allow you to build and compile software from source code:

    sudo yum -y groupinstall development

Once everything is installed, our setup is in place and we can go on to install Python 3.
Step 2 — Installing and Setting Up Python 3

CentOS is derived from RHEL (Red Hat Enterprise Linux), which has stability as its primary focus. Because of this, tested and stable versions of applications are what is most commonly found on the system and in downloadable packages, so on CentOS you will only find Python 2.

Since instead we would like to install the most current upstream stable release of Python 3, we will need to install IUS, which stands for Inline with Upstream Stable. A community project, IUS provides Red Hat Package Manager (RPM) packages for some newer versions of select software.

To install IUS, let’s install it through yum:

    sudo yum -y install https://centos7.iuscommunity.org/ius-release.rpm

Once IUS is finished installing, we can install the most recent version of Python:

    sudo yum -y install python36u

When the installation process of Python is complete, we can check to make sure that the installation was successful by checking for its version number with the python3.6 command:

    python3.6 -V

With a version of Python 3.6 successfully installed, we will receive the following output:

Output
Python 3.6.1

We will next install pip, which will manage software packages for Python:

    sudo yum -y install python36u-pip

A tool for use with Python, we will use pip to install and manage programming packages we may want to use in our development projects. You can install Python packages by typing:

    sudo pip3.6 install package_name

Here, package_name can refer to any Python package or library, such as Django for web development or NumPy for scientific computing. So if you would like to install NumPy, you can do so with the command pip3.6 install numpy.

Finally, we will need to install the IUS package python36u-devel, which provides us with libraries and header files we will need for Python 3 development:

    sudo yum -y install python36u-devel

The venv module will be used to set up a virtual environment for our development projects in the next step.
Step 3 — Setting Up a Virtual Environment

Now that we have Python installed and our system set up, we can go on to create our programming environment with venv.

Virtual environments enable you to have an isolated space on your computer for Python projects, ensuring that each of your projects can have its own set of dependencies that won’t disrupt any of your other projects.

Setting up a programming environment provides us with greater control over our Python projects and over how different versions of packages are handled. This is especially important when working with third-party packages.

You can set up as many Python programming environments as you want. Each environment is basically a directory or folder in your computer that has a few scripts in it to make it act as an environment.

Choose which directory you would like to put your Python programming environments in, or create a new directory with mkdir, as in:

    mkdir environments
    cd environments

Once you are in the directory where you would like the environments to live, you can create an environment by running the following command:

    python3.6 -m venv my_env

Essentially, this command creates a new directory (in this case called my_env) that contains a few items that we can see with the ls command:

bin include lib lib64 pyvenv.cfg

Together, these files work to make sure that your projects are isolated from the broader context of your local machine, so that system files and project files don’t mix. This is good practice for version control and to ensure that each of your projects has access to the particular packages that it needs.

To use this environment, you need to activate it, which you can do by typing the following command that calls the activate script in the bin directory:

    source my_env/bin/activate

Your prompt will now be prefixed with the name of your environment, in this case it is called my_env:


This prefix lets us know that the environment my_env is currently active, meaning that when we create programs here they will use only this particular environment’s settings and packages.

Note: Within the virtual environment, you can use the command python instead of python3.6, and pip instead of pip3.6 if you would prefer. If you use Python 3 on your machine outside of an environment, you will need to use the python3.6 and pip3.6 commands exclusively.

After following these steps, your virtual environment is ready to use.
Step 4 — Creating a Simple Program

Now that we have our virtual environment set up, let’s create a simple “Hello, World!” program. This will make sure that our environment is working and gives us the opportunity to become more familiar with Python if we aren’t already.

To do this, we’ll open up a command-line text editor such as vim and create a new file:

    vi hello.py

Once the text file opens up in our terminal window, we will have to type i to enter insert mode, and then we can write our first program:

print("Hello, World!")

Now press ESC to leave insert mode. Next, type :x then ENTER to save and exit the file.

We are now ready to run our program:

    python hello.py

The hello.py program that you just created should cause the terminal to produce the following output:

Output
Hello, World!

To leave the environment, simply type the command deactivate and you’ll return to your original directory.