Grafana Course 2

PRESENTATION
Hello!

Thank you for accessing this booklet produced by NOTO TI. Before you begin,
I would like to make a brief presentation so that you can know the origins of this.
project.

A NOTO TI emerged in the city of Jaraguá do Sul, Santa Catarina, in 2016, being
result of my experience in the sector. My name is Evandro José Zipf, I have been
mais de 18 anos no ramo de redes de computadores. Além disso, possuo Certificado
Zabbix Specialist e Professional, e ampla bagagem em Linux, Cloud Computing (AWS)
the Firewall.

My company provides Information Technology services for environments in

Cloud, Consulting in Security and Monitoring Environment. All this with the
know-how of specialized professionals who are constantly trained, ready
to provide service within high standards of quality and efficiency.

NOTO TI delivers to its clients the most modern monitoring solutions.

of environments, allowing real-time access to them from various applications
essential to decision-making. And it is with the same purpose of excellence
we prepared this booklet for you.

So, let's go, straight to the point. Good class!

nototi.com.br
 Grafana Course 3

Lesson 1: Installation

What is Grafana?

Grafana is an open source suite for real-time metric visualization and analysis through
of graphs, which allows to alert, consult and understand relevant information stored in
various locations. This way, you can monitor data using a single, intuitive interface, and
complete and streamline decision-making.

Grafana installation and integration with Zabbix

▪ Ubuntu and Debian

Package installation

1 sudo apt-get install -y adduser libfontconfig1

wgethttps://dl.grafana.com/oss/release/grafana_7.3.1_amd64.deb
# sudo dpkg -i grafana_7.3.1_amd64.deb

Restart and enable the Grafana service to start on boot

2 sudo /bin/systemctl daemon-reload

sudo /bin/systemctl enable grafana-server.service
# sudo /bin/systemctl start grafana-server.service

nototi.com.br
 Grafana Course 4

▪ Redhat and Centos

wgethttps://dl.grafana.com/oss/release/grafana-7.3.1-1.x86_64.rpm

sudo yum install grafana-7.3.1-1.x86_64.rpm

1 # sudo /bin/systemctl daemon-reload
# sudo /bin/systemctl enable grafana-server.service
# sudo /bin/systemctl start grafana-server.service

nototi.com.br
 Grafana Course 5

▪ Installation and permissions of plugins in Grafana

Installation of the plugin for integration with Zabbix

1
# grafana-cli plugins install alexanderzobnin-zabbix-app

Important: Starting from version 7 of Grafana, it is necessary to allow plugins that are not
killed. To do this, pass the name of the plugin inside the Grafana configuration file. See
how to do:

First, check the logs

2 # tail -f /var/log/grafana.log
# vim /etc/grafana/grafana.ini

Procure and align [plugins] and add the Zabbix plugin

3
allow_loading_unsigned_plugins = alexanderzobnin-zabbix-datasource

nototi.com.br
 Grafana Course 6

Do not forget to remove the ';' from the front, if it is there.

4 Salve e saia do arquivo, ereinicie o serviço do Grafana

# service grafana-server restart

Tip: Check the logs

# tail -f /var/log/grafana/grafana.log

▪ Installation via Docker

Docker is a platform written in Go for developers and system administrators that

allows the creation, execution, and sharing of applications using containers. This language of
high-performance programming provides flexibility and portability, since
packages an application and its compartments in a virtual container. Docker is present
today in projects built on the most different platforms.

Installation of Docker via Script that detects automatically

1
# curl -GsSL https://get.docker.com |sh

Add the user to the Docker group (optional) - run Docker with another user
2
# sudo usermod -aG docker noto

Check service status

3
# service docker status

nototi.com.br
 Grafana Course 7

Create the volume first. This way, the container's data remains persistent even if it is
removed. Do:
4
# docker volume create grafana-storage

Start the container:

# docker run -d \
-p 3000:3000 \
--name=grafana \
GF_SERVER_ROOT_URL=http://grafana.noto.local
5 GF_SECURITY_ADMIN_PASSWORD=noto
GF_INSTALL_PLUGINS=alexanderzobnin-zabbix-app
GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=alexanderzobnin-zabbix-
datasource
--restart unless-stopped \
-v grafana-storage:/var/lib/grafana \
grafana/grafana:7.2.0-ubuntu

Check if the container started correctly

docker logs grafana

6
Show the running container

# docker container ls

nototi.com.br
 Grafana Course 8

nototi.com.br
 Grafana Course 9

nototi.com.br
 Grafana Course 10

▪ Accessing Grafana and integrating it with Zabbix

Ready! Now, to access, check the IP address of the server you installed it on.
Grafana.

To set a static IP on Ubuntu Server 18.04, type the command:

1
show IP address

In my case here, the IP address of my machine is 192.168.98.14

To access Grafana, open a browser of your choice and type the URL:
http://seuip:3000.
2
In my case, it goes like this:

nototi.com.br
 Grafana Course 11

Ok! Access Grafana using the default username and password.

User: admin
Password: admin
3
You will be asked to register a password for the admin. Please enter one of your own.
preference.

Grafana home screen:

4 Open the plugins panel

nototi.com.br
 Grafana Course 12

5 Search for Zabbix click to access

6 Click enable

7 Access the data source and click on add new data source

nototi.com.br
 Grafana Course 13

8 Look for Zabbix

nototi.com.br
 Grafana Course 14

Settings:
Name = Give a name to the data source of your choice
9
HTTP:
URL = http://localhost/zabbix/api_jsonrpc.php

Observations:

1. The URL you should use according to the default installations of Zabbix is ip/zabbix.
2. I am assuming that you installed Grafana on the same machine where Zabbix is.
If you have installed Zabbix on another machine in your network, change localhost in the URL to the IP.
from your Zabbix. Example:http://ipdoseuzabbix/zabbix/api_jsonrpc.php

After filling in the fields above, register a username and password to connect to the API.
Zabbix.

Attention: Remember that here you need to enter the username and password of your Zabbix, and not of
Grafana. I chose the userAdmin of Zabbix. You can use any other.
10
Zabbix API details:

username: Admin
senha: zabbix

Click on Save and Test, if everything goes well, your Zabbix is already integrated with the
11 Grafana.

nototi.com.br
 Grafana Course 15

▪ grafana-cli command tip

See more at:

https://grafana.com/docs/grafana/latest/administration/cli/

Next...

Class 2: Improving Query Performance in Grafana Integrated with Zabbix

nototi.com.br
 Grafana Course 16

nototi.com.br
 Grafana Course 17

Class 2: Improving Query Performance in

Grafana Integrated with Zabbix

Segundo a documentação doGrafana Zabbix, o desempenho fica melhor em consultas maiores

when they are made directly by the database, instead of by API.

See the documentation:

https://alexanderzobnin.github.io/grafana-zabbix/reference/direct-db-connection/

So, let's set this up: create a new MySQL data source.

Add data source → MySQL

1
Attention: I am using MySQL because my Zabbix is using this database.
data. If you use Zabbix with PostgreSQL, register using the plugin
The plugins for both are native in Grafana.

Here you need to pass the credentials to connect to the Zabbix database.
you can use the same user created during the software installation, however, for security reasons, it is
it is recommended to create a read-only user access to the database. Do the following:

Log in to MySQL via console

2 # mysql -u root -p

It will ask for a password; enter your MySQL root password

Create the user Grafana with the password grafana@2020

mysql> create user grafana@localhost identified by 'grafana@2020';

nototi.com.br
 Grafana Course 18

Give read-only permission to the user grafana in the Zabbix database.

3
mysql> grant select on zabbix.* to grafana@localhost;

Ready! Open the settings of the MySQL data source and enter these credentials

Host=localhost:3306
Database=zabbix
4 User=grafana
Password=suasenha

Click on Save and Test

nototi.com.br
 Grafana Course 19

Now, edit the existing Zabbix data source, enable the direct connection to the database
5 data from Zabbix through its plugin.

nototi.com.br
 Grafana Course 20

nototi.com.br
 Grafana Course 21

6 Click on Save & Test

Done! The heavier queries can now be performed directly on the database.
7 data, and not through an API.

nototi.com.br
 Grafana Course 22

nototi.com.br
 Grafana Course 23

Class 3: Creating Simple Dashboard

In this lesson, you will learn how to create a dashboard for servers within Grafana.
Thus, you are already getting familiar with the software panels. Below is the screen that we will create.

Ato follow...

Lesson 4: Creating Automated Dashboards

nototi.com.br
 Grafana Course 24

nototi.com.br
 Grafana Course 25

Lesson 4: Creating Automated Dashboards

Imagine a structure with 100 servers and having to create a dashboard for each one.
visualize the same things. And, after everything is ready, have to update them all because of
a small modification... It’s a waste of time and money, don’t you agree?

In monitoring, you need to automate everything possible, especially in the area of

dashboards. Therefore, learn the step-by-step process to automate dashboards in Grafana.
using variable resources.

For this, we will take the same dashboard from the last class. Look:

I will give an example with two Linux servers that I have inside my lab.
only for testing. Their names: Linux01 and Linux02, and these servers are in the group of
host of Zabbix called GRAFANA.

1 Access Dashboard Settings

nototi.com.br
 Grafana Course 26

2 Go to Variables

3 Add the variable group

nototi.com.br
 Grafana Course 27

4 Click on Update and add one more variable calledhost

5 Click on Update. It will look like the screen below:

nototi.com.br
 Grafana Course 28

Go back to the dashboard screen and notice that the two variables have been created in the corner.
6 superior.

Now, to automate, it is necessary to place the variables in each created component.

7 dashboard. See:

nototi.com.br
 Grafana Course 29

Notice that the variable $grupon was added to group and the variable $host to host.
Do this for all memory components, processor, etc. After completing this
procedure, the automation is ready.
8
Alternate between the hosts, in my case Linux01 and Linux02, and see the automation in
functioning.

Next class:

Lesson 5: Automating with Repeat

nototi.com.br
 Grafana Course 30

nototi.com.br
 Grafana Course 31

Class 5: Automating with Repeat

The repeat option is another interesting automation alternative. Let's create a dashboard.
like this:

We will also create the group and host variables. Pay attention to the detail:

In the host variable, activate the options:

1 Multi-value
Include All option

nototi.com.br
 Grafana Course 32

Click on Update. Then, edit all components and select the repeat option.
2 each of them. Use the variable host as a reference.

Once the previous step is completed, just test the repetition of the components by selecting each one.
3 host in the variable host

nototi.com.br
 Grafana Course 33

4 Repeat the same components for each selected host

5 Tip: In the panel title, place the variable $host.

6 Note that the name of each host is in the title of the component.

nototi.com.br
 Grafana Course 34

Don't miss the next class:

Class 6: Repeat with Row

nototi.com.br
 Grafana Course 35

nototi.com.br
 Grafana Course 36

Class 6: Repeat with Row

Another interesting option is the use of row to separate the components within the
dashboard. With this, you can hide or show the data you want to have on the screen. Let's go to the
examples!

1 First, disable the current repetition in the components.

2 Click on add panel; then, on convert row

nototi.com.br
 Grafana Course 37

3 In the title, replace the variable $hosteRepeat with host

Click on Update
4 Ready! Select one or more hosts and see the result. Also note that now you
There is an option to hide/show data by host.

Check out: Class 7 - Regex and Functions, and take your monitoring to the next level with Grafana!

nototi.com.br
 Grafana Course 38

nototi.com.br
 Grafana Course 39

Class 7: Regex and Functions

In this lesson, you will reach an advanced level within Grafana, learning to use Regex and
Functions. Uma etapa importante no tratamento de dados. Vou mostrar alguns exemplos para
you make your dashboards much more professional.

Without treatment

With treatment

nototi.com.br
 Grafana Course 40

nototi.com.br
 Grafana Course 28

Go back to the dashboard screen and notice that the two variables have been created in the corner.
6 superior.

Now, to automate, it is necessary to place the variables in each created component.

7 dashboard. See:

nototi.com.br
 Grafana Course 42

2 Three variables appeared within the dashboard. See how the partition one is.

3 Okay, but I only want to leave the partition; I don't want used disk... Learn how to do it:

We use regex to replace and keep only the partition name. The () and the .* replace
4 all text only by what is inside the parentheses. See now how it turned out:
nototi.com.br
 Grafana Course 43

We still need to make one last adjustment to the Query; it is also necessary to add a regex.
5 inside her.

nototi.com.br
 Grafana Course 44

The query item turned out as follows:

6 Used disk space on $partition

Now it's ready! Do the tests by selecting the hosts and partitions.

Attention: Hey! What if I want to create the partition charts separated by Used and Free,
Do I have to do it one by one? No! Find out how:

7 Let's duplicate aQuery Usedem 3: Total, Used, Free

nototi.com.br
 Grafana Course 45

Asregex, the items that remained:

Total - /^Total disk space on $partition$/

8 Used disk space on $partition
Free disk space on $partition

Next step: enable to repeat the data by partition.

9 Ready! Now, just test it. Take a look at how it turned out:

nototi.com.br
 Grafana Course 46

nototi.com.br
 Grafana Course 47

Class 8: Creating Detailed and Automated Dashboards

In the same dashboard we did before, create two rows: one called DISCO
1 and another, NETWORK, leaving it as shown in this image:

2 Create one more variable called interface, as shown in the screen below:

nototi.com.br
 Grafana Course 48

Leave:
Group =$grupo
host =$host
Application =Network Interfaces
3 item =/.*/
Incoming network traffic on (.*)

Click on Update and see how it looks on the dashboard screen:

nototi.com.br
 Grafana Course 49

Now we need to adjust the Query. So, do it:

In the first Query put:

/Incoming network traffic on ($interface)/

Function: setAlias (Input $interface)

4
On Monday:

Outgoing network traffic on ($interface)

Function: setAlias (Saída $interface)

Take the opportunity to insert the variable in the component title. Therefore:

nototi.com.br
 Grafana Course 50

It's time to test! Use a host that has more than one network interface. See my
5 example environment:

nototi.com.br
 Grafana Course 51

6 Dashboard finalized, automated, and data processed

7 I wanted two interfaces. Wow! It got easy!

nototi.com.br
 Grafana Course 52

Practice now; create from this dashboard, comparing CPU consumption,

8 memory etc. Feel free, and good practice!

Next class to elevate the level of your services: Mathematics with Grafana

nototi.com.br
 Grafana Course 53

nototi.com.br
 Grafana Course 54

Lesson 9: Mathematics with Grafana

Functions and Transforms

We have some other functions and a new option in Grafana version 7, which is the part of
Transforms.You will learn to calculate data and/or transform it.

1 Imagine that you want to add the load of 1 minute with 5 and 15 to have a total. Look:

Let's duplicate the Query and use the function sum(1m)

2

nototi.com.br
 Grafana Course 55

"1m" means that I want to sum the data in the range of 1m. This is the time.
to update the load items in Zabbix. I used a regex to capture everything that
for item load, that's why the /load/.

SetAlias to make the name friendlier; this function is only Sum.

See the result:

4 Note that there are other functions: average, max, min, etc.

You can sum from any data. Let's practice with network interface. Check it out.
5 to follow...

nototi.com.br
 Grafana Course 56

6 I want to sum the incoming and outgoing traffic and put the name of the aggregate.

Hey! Here we have a new regex: /Outgoing|Incoming/

7 It means 'I want the items that contain Outgoing or Incoming'

Result:

nototi.com.br
 Grafana Course 57

Legal, isn't it? Cool, now we have a challenge for you:

How do I automate when I have multiple network interfaces and I want a

Separate chart for each one? Can I have equipment with eth0, enp0s3, etherx
etc.

8 Your client requested a separate network graph dashboard + the aggregate, but
he has several servers and/or equipment, and he doesn't want to do it one by one, he wants
something practical and automated.

Xiiiiiiii! Complicated?! Calm down, it will be fine. So, practice, try to do it without watching the
next class, in which we will show the step by step with all the commands and tips.
Agreed? Good luck and see you then!

nototi.com.br
 Curso Grafana 58

CLOSING
A NOTO TI thanks you for accessing this manual. And don't forget to
also follow our channels on digital media. Feel free to join
in contact, clarify doubts and check new content that NOTO TI provides
often.

Hug

Evandro José Zipf

blog.nototi.com.br/ /notomonitoramento/ youtube.com/c/NotoTI

nototi.com.br