Rsyslog Server with Telegram Notification using Python
Rsyslog is a syslog service that already comes built in with several linux distros, in this case we’ll be using Ubuntu. To enable this service, all we need to do is to open the listening port on port 514.
Enabling Rsyslog
Edit the rsyslog conf file
1
sudo nano /etc/rsyslog.conf
and then uncomment these lines
1
2
3
4
5
6
7
# UNCOMMENT THESE
module(load="imudp")
input(type="imudp" port="514")
# UNCOMMENT THESE
module(load="imtcp")
input(type="imtcp" port="514")
After that restart the rsyslog service
1
sudo systemctl restart rsyslog
After that go to the agent endpoint and point the syslog host to the ubuntu machine, in this case the endpoint will be a cisco router and the rsyslog server’s ip is 198.18.0.202
1
2
xe5#conf t
xe5(config)#logging host 198.18.0.202
Now the rsyslog server will start receiving syslog messages. These messages by default will be saved in /var/log/syslog
1
2
3
4
5
6
7
tail -f /var/log/syslog
Jul 22 15:48:01 198.18.0.125 1866796: *Jul 22 15:39:19.417: %OSPF-4-ERRRCV: Received invalid packet: mismatched area ID from backbone area from 2.2.45.4, GigabitEthernet2
Jul 22 15:48:06 198.18.0.125 1866797: *Jul 22 15:39:24.737: %OSPF-4-ERRRCV: Received invalid packet: mismatched area ID from backbone area from 2.2.23.2, GigabitEthernet2
Jul 22 15:48:11 198.18.0.125 1866798: *Jul 22 15:39:29.824: %OSPF-4-ERRRCV: Received invalid packet: mismatched area ID from backbone area from 2.2.12.1, GigabitEthernet2
Jul 22 15:48:17 198.18.0.125 1866799: *Jul 22 15:39:35.884: %OSPF-4-ERRRCV: Received invalid packet: mismatched area ID from backbone area from 2.2.34.3, GigabitEthernet2
Jul 22 15:48:24 198.18.0.125 1866800: *Jul 22 15:39:42.220: %OSPF-4-ERRRCV: Received invalid packet: mismatched area ID from backbone area from 2.2.45.5, GigabitEthernet2
Change the output path and Filter the messages
First let’s point our endpoint’s syslog messages so it doesn’t get mixed with the rest of log messages in the default file, to do that we’re gonna create a new configuration file
1
sudo nano /etc/rsyslog.d/10-helena.conf
And this is how the conf looks like
1
2
3
4
5
6
7
8
9
$template RouterLog, "/opt/rsyslog/helena.log"
:msg, contains, "-4-" ~
:msg, contains, "-5-" ~
:msg, contains, "-6-" ~
:msg, contains, "-7-" ~
:fromhost-ip, isequal, "198.18.0.125" -?RouterLog
& stop
- $template RouterLog, “/opt/rsyslog/helena.log” is where the logs will be stored.
- :msg, contains, “-4-“ ~g” will filter out the logs that contains the pattern “-4-“ so it will be discarded.
For cicso devices, there are 8 levels of severity ranging from 0 to 7 with lower means higher severity. Here i only want to collect the logs for severity 0 to 3, thus filtering the rest with these lines of rules.
- lastly, the line :fromhost-ip, isequal, “198.18.0.125” -?RouterLog will filter only logs messages from this particular IP to be stored here. The rest will still go to the default log path.
Don’t forget to create the directory where the logs will be stored
1
2
sudo mkdir /opt/rsyslog/
sudo chmod 777 /opt/rsyslog/
Restart the rsyslog service for this to take effect
1
sudo systemctl restart rsyslog
After that we’re gonna start seeing some logs being collected in the new path
1
2
3
4
5
sudo tail -n 100 -f /opt/rsyslog/helena.log
Jul 22 16:11:34 198.18.0.125 1867043: *Jul 22 16:02:52.513: %LINK-3-UPDOWN: Interface GigabitEthernet2, changed state to up
Jul 22 16:13:13 198.18.0.125 1867071: *Jul 22 16:04:32.416: %LINK-3-UPDOWN: Interface GigabitEthernet3, changed state to up
Jul 22 16:13:14 198.18.0.125 1867073: *Jul 22 16:04:32.515: %LINK-3-UPDOWN: Interface GigabitEthernet2, changed state to up
That’s pretty much it for the rsyslog part. Now we create a python script to send an alert to telegram.
Sending alert to Telegram
Let’s code the program to read the log messages and send it to telegram.
1
sudo nano /opt/rsyslog/sendtelegram.py
First we set up some dependecies, like importing the modules needed. Here we also need to import asyncio because for some reason it won’t send the telegram message without it on linux. Works fine without it on any other OS so idk. . .
1
2
3
4
5
6
import telegram
import asyncio
TELEGRAM_BOT_TOKEN = 'your bot token'
TELEGRAM_CHAT_ID = 'your chat id'
LOG_PATH = "/opt/rsyslog/helena.log"
Now we create the logic to read the logs message from the rsyslog output file. If there’s detected at least one line of log there, we return True with the logs message.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def check_log_line():
message = ""
try:
with open(LOG_PATH, 'r') as file:
lines = file.readlines()
if lines:
for line in lines:
message = line+"\n"
return True,message
else:
message = "Log empty"
return False,message
except:
message = "Error exception"
return False,message
This part is where we tell the code to send the message to telegram. It’s pretty straightforward so no need that much of an explanation.
1
2
3
4
5
6
7
async def send_telegram(message):
try:
bot = telegram.Bot(token=TELEGRAM_BOT_TOKEN)
await bot.send_message(chat_id=TELEGRAM_CHAT_ID, text=message)
print("* Telegram message sent")
except Exception as e:
print(f"* Failed sending telegram message {e}")
And lastly this is where we call all the functions we created above. If the log_line_check returns True, we send the telegram message.
1
2
3
4
5
6
log_line_check, message = check_log_line()
if log_line_check:
asyncio.run(send_telegram(message))
else:
print("No logs detected.")
After that, just run the program
1
sudo python3 /opt/rsyslog/sendtelegram.py
The message should be sent to your chat or group, depending what chat id you provided.
Extra features
This is pretty basic and barebone you might think, but at least we managed to make it work first. After that, you can play around with other services to make this more robust and smarter, for example you can incomporate logrotate to automatically truncate the syslog messages as it is being sent over telegram, and run everything automatically every certain amount of time with cronjob.
If you’re interested to explore more, i built something more or less like that here on this Github Repository