www.it-connect.fr
Notifications with Graylog – How do I send e-mail alerts?
Table of Contents
I. Introduction
Graylog can store, index and analyze your device logs. And that's not all: it can also send you a notification when a specific event is detected. The solution can send notifications by e-mail, on Slack, on Microsoft Teams, as well as other methods.
In this tutorial, we'll learn how to configure Graylog to send e-mails via an SMTP server, and then see how to set up a first alert. Here, we'll take the example of receiving an e-mail when a failed SSH connection is detected on a host.
We assume that the Graylog server is in place and that a Linux machine sends its logs to this server via Rsyslog.
Other articles on the Graylog theme are available on our site, including :
- Deploy Graylog on Debian 12 to easily centralize and analyze your logs
- Send Linux logs to Graylog with rsyslog
- Send Windows logs to Graylog with NXLog
II. Graylog: SMTP configuration for sending e-mails
The first step is to adjust Graylog's configuration so that it can send e-mails. You'll see how to configure the SMTP server, port number, sender e-mail address, etc. in Graylog's "server.conf" configuration file.
Connect to your Graylog server terminal to edit the configuration file:
sudo nano /etc/graylog/server/server.confWithin this file, the parameters for configuring the SMTP server are already present, but they are commented out (and therefore inactive). Sending e-mails is disabled by default, which is normal since it's up to you to configure notifications according to your environment. The parameters to be edited have a name starting with "transport_email".
You need to enable e-mail delivery (transport_email_enabled) and configure the various parameters to specify the SMTP server name (transport_email_hostname), the port number to be used (transport_email_port) and the credentials. If you wish to authenticate via Gmail (Google), you can use an application password (recommended), otherwise adapt to your environment. Specify a user name (transport_email_auth_username), a password (transport_email_auth_password) and a sender e-mail address to be used by Graylog (transport_email_from_email).
transport_email_enabled = true
transport_email_hostname = smtp.gmail.com
transport_email_port = 587
transport_email_use_auth = true
transport_email_auth_username = GoogleAccount@gmail.com
transport_email_auth_password = YourPassword
transport_email_from_email = GoogleAccount@gmail.com
transport_email_socket_connection_timeout = 10s
transport_email_socket_timeout = 10sTo use a secure connection based on TLS, also activate this parameter (it is located a little further down):
transport_email_use_tls = trueGo to the configuration file to configure this parameter:
transport_email_web_interface_url = http://192.168.10.220:9000You need to define the IP address or domain name and port for accessing Graylog. This is the URL that will be used in e-mail alerts to redirect you directly to the Graylog portal.
When this is done, save the changes and restart Graylog :
sudo systemctl restart graylog-server.serviceThe rest of the configuration is carried out via the Graylog web interface.
III. Graylog: configuring e-mail alerts
A. Create a new Stream
Now we're going to create a new Stream to retrieve the log messages we need. In this case, messages referring to a failed SSH connection. Setting up a new stream is no coincidence: an alert is set up in relation to a stream.
From Graylog, click on the "Streams" menu and create a new stream. Name this stream, for example "SSH connection errors". Keep the index named "Default index set", unless you already have a specific stream for Linux logs (this is the case if you've followed the previous tutorials). As far as I'm concerned, the index named "Linux Index" will be selected.
In the list of Streams, identify the one you've just created, then at the end of the line, click on "More" and then on "Manage Rules".
Add a new rule using the "Add stream rule" button. The idea is to create conditions to isolate messages corresponding to unsuccessful SSH connections. I invite you to create 2 conditions:
- A condition to indicate that the message must contain the string "Failed password".
- A condition to indicate that the message must correspond to the "sshd" application.
This should give you the following result:
Confirm with the "I'm done!" button. Your new Stream will be paused, as shown by the "Paused" message in the button. Click on it to make it active.
Once that's done, move on to the next step.
B. Create a new notification type
In the Graylog menu, click on the entry entitled "Alerts", then switch to the "Notifications" tab and click on "Create notification". This will create an e-mail configuration type, with a personalized message.
A new wizard appears. Start by giving this notification a name. It's possible to use one notification template for several alerts, so you may want to use a more generic name. Also choose the notification type "Email Notification".
You can then define the sender's e-mail address (Sender). If you leave it empty, the address defined in the "server.conf" file will be used. On the other hand, you must define the recipient(s), via the "Email recipient(s)" option.
A little further down the page, you have access to the template corresponding to the body of the notification, i.e. its content. This is not Graylog's strong point, as defining a template is not easy... And it's generally more convenient to click on the link in the e-mail to perform the analysis on Graylog. For the moment, click on "Execute test notification" to test sending an e-mail.
If the configuration is correct, a validation message is displayed. If not, check the parameters on this page, as well as those defined in the "server.conf" file.
Here is the e-mail I received for this test:
When it's ready, move on to the next step.
C. Create a new alert
The last step in the configuration process is to create a new event. Still in the "Alerts" section of the menu, click on "Event Definitions" to access the "Create event definition" button.
Name this alert, for example "SSH connection error - Linux" and continue.
In the "Condition" step, select "Filter & Aggregation" as the type. This will bring up additional fields.
You need to configure a filter. You can specify "Failed password" as "Search query" and select the Stream created earlier. I would also like to draw your attention to two parameters:
- Search within the last: search within last-minute events.
- Execute search every: execute the search for matching events every minute
So if there's a failed SSH connection on one of your servers, you'll be alerted within a minute of the event occurring on the server.
Continue, skip the "Fields" step and switch to "Notifications". Here, select the notification template you've created and confirm with the "Add notification" button.
Finally, click on the "Summary" step and confirm with the "Create event definition" button. The configuration is now complete!
D. Test the alert
Now we're going to test our new configuration to see if we're receiving the e-mail alert... Nothing could be simpler: just simulate an SSH connection on a server, specifying a wrong password.
A few seconds later, an e-mail should arrive in your inbox. By clicking on the link on the "Alert Replay" line, you can view all the events associated with this alert, for analysis.
The configuration is now operational!
IV. Conclusion
By following this tutorial, you'll be able to create your first Graylog alert. It's not the most practical function in Graylog, but it's important to know that it exists and that it's a good way of receiving notification when a suspicious event occurs.
