xavier
/
internet-sla-monitoring
mirror of https://github.com/xeviff/internet-sla-monitoring.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: xavier:master
Branches Tags
xavier:master
xavier:6-starlink
xavier:home
xavier:1.1.1
xavier:1.1.0
xavier:1.0.0
..
compare: xavier:1.0.0
Branches Tags
xavier:master
xavier:6-starlink
xavier:home
xavier:1.1.1
xavier:1.1.0
xavier:1.0.0

No commits in common. "master" and "1.0.0" have entirely different histories.
master ... 1.0.0

11 changed files with 636 additions and 1190 deletions
Show Stats Expand all files Collapse all files

2
.gitignore vendored
View File

@ -1,2 +0,0 @@
.DS_Store

62
README.md
View File

@ -1,28 +1,60 @@
# Internet Service Level Agreement Monitoring Docker Stack with Prometheus, Grafana and a small python ports checker.
# A Docker Stack which Monitors your home network
> To learn more about the python ports checker use, please visit the related python project [here](https://github.com/xeviff/pyOpenPortsChecker4Prometheus)
> This repository is a fork from [maxandersen/internet-monitoring](https://github.com/maxandersen/internet-monitoring), tailored for use on a Raspberry Pi. It has only been tested on a Raspberry Pi 4 running Pi OS 64-bit beta.
Stand-up a Docker [Prometheus](http://prometheus.io/) stack containing Prometheus, Grafana with [blackbox-exporter](https://github.com/prometheus/blackbox_exporter), and [speedtest-exporter](https://github.com/MiguelNdeCarvalho/speedtest-exporter) to collect and graph home Internet reliability and throughput.
Here's a quick start to stand-up a Docker [Prometheus](http://prometheus.io/) stack containing Prometheus, Grafana with [blackbox-exporter](https://github.com/prometheus/blackbox_exporter) and [speedtest-exporter](https://github.com/stefanwalther/speedtest-exporter) to collect and graph home network connections and speed.
## This is a fork
## Pre-requisites
Please, be aware this is an improvement based on an already existing project, so if you are interested on more details about its components, then I encourage you to take as first reference the original project [here](https://github.com/geerlingguy/internet-monitoring)
Before we get started installing the Prometheus stack. Ensure you install the latest version of docker and [docker-compose](https://docs.docker.com/compose/install/) on your Docker host machine. This has been tested with Docker for Mac and Synology and it works.
## Configurations
# Quick Start
That's easy, check the both projects configuration steps :wink:
Another important thing is the threshold for the internet speed. In my case I should have 1GB symmetrical and if one day I'll have this real speed, the colors would get the green color (but I don't expect this will happen with fibracat.cat :triumph:). Anyway if you had 600 Mbps for example, you'd need to adjust the Grafana's dashboard thresholds to have the green color when the values reaches 600 instead of 1000.
But it's pretty easy, you (as grafana's administrator) edit the panels and change these values:
```
git clone https://github.com/geerlingguy/internet-monitoring
cd internet-monitoring
docker-compose up
```
![image](https://user-images.githubusercontent.com/73612508/176744529-b2fd6d5a-c14e-4924-99b6-80959b964947.png)
Goto [http://localhost:3030/d/o9mIe_Aik/internet-connection](http://localhost:3030/d/o9mIe_Aik/internet-connection) (change `localhost` to your docker host ip/name).
## Configuration
To change what hosts you ping you change the `targets` section in [/prometheus/pinghosts.yaml](./prometheus/pinghosts.yaml) file.
## Expected result
For speedtest the only relevant configuration is how often you want the check to happen. It is at 5 minutes by default which might be too much if you have limit on downloads. This is changed by editing `scrape_interval` under `speedtest` in [/prometheus/prometheus.yml](./prometheus/prometheus.yml).
This is how it looks like in my NAS:
Once configurations are done let's start it up. From the /prometheus project directory run the following command:
<center><img src="images/dashboard.jpeg" width="4600" heighth="500"></center>
$ docker-compose up -d
<br>
The interesting thing is you can adjust the time visibility and get its average, for example the last 24h or even last week stats, etc.
That's it. docker-compose builds the entire Grafana and Prometheus stack automagically.
The Grafana Dashboard is now accessible via: `http://<Host IP Address>:3030` for example http://localhost:3030
username - admin
password - wonka (Password is stored in the `config.monitoring` env file)
The DataSource and Dashboard for Grafana are automatically provisioned.
If all works it should be available at http://localhost:3030/d/o9mIe_Aik/internet-connection - if no data shows up try change the timeduration to something smaller.
<center><img src="images/dashboard.png" width="4600" heighth="500"></center>
## Interesting urls
Note: replace `localhost` with your docker host ip/name if not running this locally.
http://localhost:9090/targets shows status of monitored targets as seen from prometheus - in this case which hosts being pinged and speedtest. note: speedtest will take a while before it shows as UP as it takes about 30s to respond.
http://localhost:9090/graph?g0.expr=probe_http_status_code&g0.tab=1 shows prometheus value for `probe_http_status_code` for each host. You can edit/play with additional values. Useful to check everything is okey in prometheus (in case Grafana is not showing the data you expect).
http://localhost:9115 blackbox exporter endpoint. Lets you see what have failed/succeded.
http://localhost:9798/metrics speedtest exporter endpoint. Does take about 30 seconds to show its result as it runs an actual speedtest when requested.
## Thanks and a disclaimer
Thanks to @vegasbrianc work on making a [super easy docker](https://github.com/vegasbrianc/github-monitoring) stack for running prometheus and grafana.
I also want to disclaim that Prometheus aren't really (currently) intended for this kind of blackbox/external monitoring and this setup is not in anyway secured. Thus only use this for inspiration and do not blame me if someone hacks this and figure out what your real internet speed is :)

20
docker-compose.yml
View File

@ -11,7 +11,6 @@ networks:
services:
prometheus:
image: prom/prometheus:v2.25.2
restart: always
volumes:
- ./prometheus/:/etc/prometheus/
- prometheus_data:/prometheus
@ -25,13 +24,11 @@ services:
links:
- ping:ping
- speedtest:speedtest
- portschecker:portschecker
networks:
- back-tier
grafana:
image: grafana/grafana
restart: always
volumes:
- grafana_data:/var/lib/grafana
- ./grafana/provisioning/:/etc/grafana/provisioning/
@ -53,7 +50,6 @@ services:
ports:
- 9115:9115
image: prom/blackbox-exporter
restart: always
volumes:
- ./blackbox/config:/config
command:
@ -90,19 +86,3 @@ services:
- "^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)"
networks:
- back-tier
portschecker:
tty: true
stdin_open: true
image: xeviff/python-prometheus-canyouseeme
volumes:
- ./pyPrometheusPortChecker/config:/config
depends_on:
- speedtest
expose:
- 9116
ports:
- 9116:9116
restart: always
networks:
- back-tier

7
grafana/config.monitoring
View File

@ -1,4 +1,3 @@
GF_SECURITY_ADMIN_PASSWORD=grafana123
GF_USERS_ALLOW_SIGN_UP=true
GF_INSTALL_PLUGINS=flant-statusmap-panel,ae3e-plotly-panel
GF_AUTH_ANONYMOUS_ENABLED=true
GF_SECURITY_ADMIN_PASSWORD=wonka
GF_USERS_ALLOW_SIGN_UP=false
GF_INSTALL_PLUGINS=flant-statusmap-panel

1129
grafana/provisioning/dashboards/internet-SLA-monitoring.json
View File

File diff suppressed because it is too large Load Diff

568
grafana/provisioning/dashboards/ping-speed-stats.json Normal file
View File

@ -0,0 +1,568 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"limit": 100,
"name": "Annotations & Alerts",
"showIn": 0,
"type": "dashboard"
}
]
},
"editable": true,
"gnetId": null,
"graphTooltip": 0,
"links": [],
"panels": [
{
"cacheTimeout": null,
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"custom": {},
"mappings": [
{
"id": 0,
"op": "=",
"text": "N/A",
"type": 1,
"value": "null"
}
],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "#299c46",
"value": null
},
{
"color": "blue",
"value": 100
}
]
},
"unit": "bps"
},
"overrides": []
},
"gridPos": {
"h": 9,
"w": 6,
"x": 0,
"y": 0
},
"id": 8,
"interval": null,
"links": [],
"maxDataPoints": 100,
"options": {
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"mean"
],
"fields": "",
"values": false
},
"showThresholdLabels": false,
"showThresholdMarkers": true,
"text": {}
},
"pluginVersion": "7.4.5",
"targets": [
{
"expr": "speedtest_download_bits_per_second{}",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "",
"refId": "A"
}
],
"title": "Download",
"type": "gauge"
},
{
"cacheTimeout": null,
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"custom": {},
"mappings": [
{
"id": 0,
"op": "=",
"text": "N/A",
"type": 1,
"value": "null"
}
],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "blue",
"value": 20
}
]
},
"unit": "bps"
},
"overrides": []
},
"gridPos": {
"h": 9,
"w": 6,
"x": 6,
"y": 0
},
"id": 10,
"interval": null,
"links": [],
"maxDataPoints": 100,
"options": {
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"mean"
],
"fields": "",
"values": false
},
"showThresholdLabels": false,
"showThresholdMarkers": true,
"text": {}
},
"pluginVersion": "7.4.5",
"targets": [
{
"expr": "speedtest_upload_bits_per_second{}",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "",
"refId": "A"
}
],
"title": "Upload",
"type": "gauge"
},
{
"cacheTimeout": null,
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"custom": {},
"decimals": 2,
"mappings": [
{
"id": 0,
"op": "=",
"text": "N/A",
"type": 1,
"value": "null"
}
],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "ms"
},
"overrides": []
},
"gridPos": {
"h": 9,
"w": 12,
"x": 12,
"y": 0
},
"id": 12,
"interval": null,
"links": [],
"maxDataPoints": 100,
"options": {
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"mean"
],
"fields": "",
"values": false
},
"showThresholdLabels": false,
"showThresholdMarkers": true,
"text": {}
},
"pluginVersion": "7.4.5",
"targets": [
{
"expr": "speedtest_ping_latency_milliseconds",
"format": "time_series",
"instant": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "",
"refId": "A"
}
],
"title": "speedtest ping",
"type": "gauge"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"custom": {},
"unit": "bps"
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 12,
"x": 0,
"y": 9
},
"hiddenSeries": false,
"id": 6,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 5,
"points": false,
"renderer": "flot",
"repeat": null,
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "speedtest_download_bits_per_second{}",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "",
"refId": "A"
},
{
"expr": "speedtest_upload_bits_per_second{}",
"hide": false,
"interval": "",
"legendFormat": "",
"refId": "B"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "speedtest",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "bps",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"cards": {
"cardHSpacing": 2,
"cardMinWidth": 5,
"cardRound": null,
"cardVSpacing": 2
},
"color": {
"cardColor": "#b4ff00",
"colorScale": "sqrt",
"colorScheme": "interpolateGnYlRd",
"defaultColor": "#757575",
"exponent": 0.5,
"mode": "spectrum",
"thresholds": []
},
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"gridPos": {
"h": 9,
"w": 12,
"x": 12,
"y": 9
},
"highlightCards": true,
"id": 14,
"legend": {
"show": false
},
"links": [],
"nullPointMode": "as empty",
"pageSize": 15,
"seriesFilterIndex": -1,
"statusmap": {
"ConfigVersion": "v1"
},
"targets": [
{
"expr": "probe_success",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "{{instance}}",
"refId": "A"
}
],
"title": "Panel Title",
"tooltip": {
"extraInfo": "",
"freezeOnClick": true,
"items": [],
"show": true,
"showExtraInfo": false,
"showItems": false
},
"type": "flant-statusmap-panel",
"useMax": true,
"usingPagination": false,
"xAxis": {
"show": true
},
"yAxis": {
"maxWidth": -1,
"minWidth": -1,
"show": true
},
"yAxisSort": "metrics",
"yLabel": {
"delimiter": "",
"labelTemplate": "",
"usingSplitLabel": false
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 24,
"x": 0,
"y": 18
},
"hiddenSeries": false,
"id": 4,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"maxPerRow": 12,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 5,
"points": false,
"renderer": "flot",
"repeat": "host",
"repeatDirection": "h",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(probe_http_duration_seconds) by (instance)",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "{{instance}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "http duration",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
}
],
"schemaVersion": 27,
"style": "dark",
"tags": [
"speedtest",
"ping"
],
"templating": {
"list": []
},
"time": {
"from": "now-30m",
"to": "now"
},
"timepicker": {
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
]
},
"timezone": "browser",
"title": "internet connection",
"uid": "o9mIe_Aik",
"version": 1
}

BIN
images/dashboard.jpeg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 381 KiB

BIN
images/dashboard.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 196 KiB

5
prometheus/pinghosts.yaml
View File

@ -1,8 +1,7 @@
- targets: # url;humanname;routing;switch
- http://www.google.com/;google.com;external;internetbox
#- https://github.com/;github.com;external;internetbox
- https://www.apple.com/;apple.com;external;internetbox
- https://www.fibracat.cat/;fibracat.cat;external;internetbox
- https://github.com/;github.com;external;internetbox
- https://www.jeffgeerling.com;jeffgeerling.com;external;internetbox
# - http://192.168.1.1;internetbox;local;internetbox
# - http://192.168.1.42;blackdata;local;orbirouter-attic
# - http://192.168.1.101;orbirouter;local;orbirouter

23
prometheus/prometheus.yml
View File

@ -1,30 +1,43 @@
# my global config
global:
scrape_interval: 15s # By default, scrape targets every 15 seconds.
evaluation_interval: 15s # By default, scrape targets every 15 seconds.
# scrape_timeout is set to the global default (10s).
# Attach these labels to any time series or alerts when communicating with
# external systems (federation, remote storage, Alertmanager).
external_labels:
monitor: 'Alertmanager'
# Load and evaluate rules in this file every 'evaluation_interval' seconds.
rule_files:
- 'alert.rules'
# - "first.rules"
# - "second.rules"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
- job_name: 'prometheus'
scrape_interval: 60s
# Override the global default and scrape targets from this job every 5 seconds.
scrape_interval: 5s
static_configs:
- targets: ['localhost:9090']
- job_name: 'speedtest'
metrics_path: /metrics
scrape_interval: 30m
scrape_interval: 5m
scrape_timeout: 60s # running speedtest needs time to complete
static_configs:
- targets: ['speedtest:9798']
- job_name: 'ping'
metrics_path: /probe
scrape_interval: 60s
scrape_interval: 5s
params:
module: [http_2xx] # Look for a HTTP 200 response.
file_sd_configs:
@ -55,7 +68,3 @@ scrape_configs:
- job_name: 'nodeexp'
static_configs:
- targets: ['nodeexp:9100']
- job_name: 'python-portschecker'
static_configs:
- targets: ['portschecker:9116']

10
pyPrometheusPortChecker/config/config.yml
View File

@ -1,10 +0,0 @@
service_port: 9116
scrape_frequency: 3600
host_to_check: yourdomain.is
ports_tocheck:
- 25
- 80
- 443
- 32400
- 22
- 51413