InfluxDB with Grafana

InfluxDB is a time-series database used to store and query network traffic data, capturing metrics like packet counts, latency, and bandwidth usage over time. The data is collected via APIs, which send real-time traffic metrics to InfluxDB. Grafana is then used to visualize the data from InfluxDB, creating interactive dashboards for monitoring network performance and identifying issues.

Installing InfluxDB

We’ll use docker to install InfluxDB as well as Grafana later on, make sure Docker and Docker Compose are installed

Next prepare the Docker Compose file to install the InfluxDB

services:
  influxdb:
    image: influxdb:latest
    ports:
      - '8086:8086'
    volumes:
      - influxdb-storage:/var/lib/influxdb
    environment:
      - DOCKER_INFLUXDB_INIT_MODE=setup
      - DOCKER_INFLUXDB_INIT_USERNAME=helena
      - DOCKER_INFLUXDB_INIT_PASSWORD=helena123
      - DOCKER_INFLUXDB_INIT_ORG=helena-gg
      - DOCKER_INFLUXDB_INIT_BUCKET=stealthwatch

volumes:
  influxdb-storage:

Run “docker-compose up” to execute it

After installation finishes, run “docker ps” to see the InfluxDB running as a container

And the Web UI should be accessible via the browser on port 8086

Creating Bucket

We’re gonna store our data in a bucket named “stealthwatch”

We’re also gonna insert the data to our bucket using API, so we need to generate API token to be used in our program

Runnig the Python Program

Here’s the code snipped used to insert data to InfluxDB, the full program is collecting data from Cisco Stealthwatch through API and then send it to InfluxDB also using API. The code can be here

inf_url = CONFIG['influx']
inf_token = CONFIG['influx_token']
inf_org = CONFIG['influx_org']
inf_bucket = CONFIG['influx_bucket']

client = InfluxDBClient(url=inf_url, token=inf_token, org=inf_org)
write_api = client.write_api(write_options=SYNCHRONOUS)

points = []

for flow in flows:
    flow_id = flow['id']
    flow_time = flow['statistics']['firstActiveTime']
    flow_protocol = flow["protocol"]
    flow_src = flow['subject']['ipAddress']
    flow_src_port = flow['subject']['portProtocol']['port']
    flow_dst = flow['peer']['ipAddress']
    flow_dst_port = flow['peer']['portProtocol']['port']
    flow_byte = flow['statistics']['byteCount']
    flow_packet = flow['statistics']['packetCount']
    timestamp_ns = int(datetime.strptime(flow_time, "%Y-%m-%dT%H:%M:%S.%f%z").timestamp() * 1e9)

    point = (
        Point("network_flow")
        .field("id", flow_id)
        .field("byte", flow_byte)
        .field("packet", flow_packet)
        .tag("protocol", flow_protocol)
        .tag("src_ip", flow_src)
        .tag("dst_ip", flow_dst)
        .tag("src_port", flow_src_port)
        .tag("dst_port", flow_dst_port)
        .time(timestamp_ns, WritePrecision.NS)
    )

    points.append(point)

write_api.write(bucket=inf_bucket, org=inf_org, record=points)
client.close()

And when the is runnning, its inserting flows / network traffic into the database on InfluxDB

Back on InfluxDB, we can see the data is now in the bucket

Installing Grafana

We will also use Docker Compose to install Grafana on the same linux server, to do that we’ll modify the config to include Grafana

services:
  influxdb:
    image: influxdb:latest
    ports:
      - '8086:8086'
    volumes:
      - influxdb-storage:/var/lib/influxdb
    environment:
      - DOCKER_INFLUXDB_INIT_MODE=setup
      - DOCKER_INFLUXDB_INIT_USERNAME=helena
      - DOCKER_INFLUXDB_INIT_PASSWORD=helena123
      - DOCKER_INFLUXDB_INIT_ORG=helena-gg
      - DOCKER_INFLUXDB_INIT_BUCKET=stealthwatch

  grafana:
    image: grafana/grafana-oss:latest
    ports:
      - '3000:3000'
    depends_on:
      - influxdb
    environment:
      - GF_SECURITY_ADMIN_USER=helena
      - GF_SECURITY_ADMIN_PASSWORD=helena123
    volumes:
      - grafana-storage:/var/lib/grafana

volumes:
  influxdb-storage:
  grafana-storage:

Then run “docker-compose up” to start Grafana alongside the already running InfluxDB

Command “docker ps” verifies it

And Grafana should also be up and accessible on port 3000

Connecting Grafana to InfluxDB

Next lets add InfluxDB as a Data Source, here because we’re running InfluxDB v2.7 (the latest stable at the time writing this), we cannot use SQL as the Query Language which will only available on v3.x. So baiscally we’re stuck with Flux

And after some Flux Querying, the data from InfluxDB now can be visualized on Grafana