xavier
/
internet-sla-monitoring
mirror of https://github.com/xeviff/internet-sla-monitoring.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: xavier:master
Branches Tags
xavier:master
xavier:6-starlink
xavier:home
xavier:1.1.1
xavier:1.1.0
xavier:1.0.0
..
compare: xavier:1.1.1
Branches Tags
xavier:master
xavier:6-starlink
xavier:home
xavier:1.1.1
xavier:1.1.0
xavier:1.0.0

No commits in common. "master" and "1.1.1" have entirely different histories.
master ... 1.1.1

13 changed files with 2861 additions and 1183 deletions
Show Stats Expand all files Collapse all files

2
.gitignore vendored
View File

@ -1,2 +0,0 @@
.DS_Store

60
README.md
View File

@ -1,28 +1,60 @@
# Internet Service Level Agreement Monitoring Docker Stack with Prometheus, Grafana and a small python ports checker.
# Internet Monitoring Docker Stack with Prometheus + Grafana
> To learn more about the python ports checker use, please visit the related python project [here](https://github.com/xeviff/pyOpenPortsChecker4Prometheus)
> This repository is a fork from [maxandersen/internet-monitoring](https://github.com/maxandersen/internet-monitoring), tailored for use on a Raspberry Pi. It has only been tested on a Raspberry Pi 4 running Pi OS 64-bit beta.
Stand-up a Docker [Prometheus](http://prometheus.io/) stack containing Prometheus, Grafana with [blackbox-exporter](https://github.com/prometheus/blackbox_exporter), and [speedtest-exporter](https://github.com/MiguelNdeCarvalho/speedtest-exporter) to collect and graph home Internet reliability and throughput.
## This is a fork
## Pre-requisites
Please, be aware this is an improvement based on an already existing project, so if you are interested on more details about its components, then I encourage you to take as first reference the original project [here](https://github.com/geerlingguy/internet-monitoring)
Make sure Docker and [Docker Compose](https://docs.docker.com/compose/install/) are installed on your Docker host machine.
## Configurations
## Quick Start
That's easy, check the both projects configuration steps :wink:
Another important thing is the threshold for the internet speed. In my case I should have 1GB symmetrical and if one day I'll have this real speed, the colors would get the green color (but I don't expect this will happen with fibracat.cat :triumph:). Anyway if you had 600 Mbps for example, you'd need to adjust the Grafana's dashboard thresholds to have the green color when the values reaches 600 instead of 1000.
But it's pretty easy, you (as grafana's administrator) edit the panels and change these values:
```
git clone https://github.com/geerlingguy/internet-monitoring
cd internet-monitoring
docker-compose up -d
```
![image](https://user-images.githubusercontent.com/73612508/176744529-b2fd6d5a-c14e-4924-99b6-80959b964947.png)
Go to [http://localhost:3030/d/o9mIe_Aik/internet-connection](http://localhost:3030/d/o9mIe_Aik/internet-connection) (change `localhost` to your docker host ip/name).
## Configuration
To change what hosts you ping you change the `targets` section in [/prometheus/pinghosts.yaml](./prometheus/pinghosts.yaml) file.
## Expected result
For speedtest the only relevant configuration is how often you want the check to happen. It is at 30 minutes by default which might be too much if you have limit on downloads. This is changed by editing `scrape_interval` under `speedtest` in [/prometheus/prometheus.yml](./prometheus/prometheus.yml).
This is how it looks like in my NAS:
Once configurations are done, run the following command:
<center><img src="images/dashboard.jpeg" width="4600" heighth="500"></center>
$ docker-compose up -d
<br>
The interesting thing is you can adjust the time visibility and get its average, for example the last 24h or even last week stats, etc.
That's it. docker-compose builds the entire Grafana and Prometheus stack automagically.
The Grafana Dashboard is now accessible via: `http://<Host IP Address>:3030` for example http://localhost:3030
username - admin
password - wonka (Password is stored in the `config.monitoring` env file)
The DataSource and Dashboard for Grafana are automatically provisioned.
If all works it should be available at http://localhost:3030/d/o9mIe_Aik/internet-connection - if no data shows up try change the timeduration to something smaller.
<center><img src="images/dashboard.png" width="4600" heighth="500"></center>
## Interesting urls
http://localhost:9090/targets shows status of monitored targets as seen from prometheus - in this case which hosts being pinged and speedtest. note: speedtest will take a while before it shows as UP as it takes about 30s to respond.
http://localhost:9090/graph?g0.expr=probe_http_status_code&g0.tab=1 shows prometheus value for `probe_http_status_code` for each host. You can edit/play with additional values. Useful to check everything is okey in prometheus (in case Grafana is not showing the data you expect).
http://localhost:9115 blackbox exporter endpoint. Lets you see what have failed/succeded.
http://localhost:9798/metrics speedtest exporter endpoint. Does take about 30 seconds to show its result as it runs an actual speedtest when requested.
## Thanks and a disclaimer
Thanks to @maxandersen for making the original project this fork is based on.
Thanks to @vegasbrianc work on making a [super easy docker](https://github.com/vegasbrianc/github-monitoring) stack for running prometheus and grafana.
This setup is not secured in any way, so please only use on non-public networks, or find a way to secure it on your own.

17
docker-compose.yml
View File

@ -25,7 +25,6 @@ services:
links:
- ping:ping
- speedtest:speedtest
- portschecker:portschecker
networks:
- back-tier
@ -90,19 +89,3 @@ services:
- "^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)"
networks:
- back-tier
portschecker:
tty: true
stdin_open: true
image: xeviff/python-prometheus-canyouseeme
volumes:
- ./pyPrometheusPortChecker/config:/config
depends_on:
- speedtest
expose:
- 9116
ports:
- 9116:9116
restart: always
networks:
- back-tier

5
grafana/config.monitoring
View File

@ -1,4 +1,3 @@
GF_SECURITY_ADMIN_PASSWORD=grafana123
GF_USERS_ALLOW_SIGN_UP=true
GF_SECURITY_ADMIN_PASSWORD=wonka
GF_USERS_ALLOW_SIGN_UP=false
GF_INSTALL_PLUGINS=flant-statusmap-panel,ae3e-plotly-panel
GF_AUTH_ANONYMOUS_ENABLED=true

1129
grafana/provisioning/dashboards/internet-SLA-monitoring.json
View File

File diff suppressed because it is too large Load Diff

560
grafana/provisioning/dashboards/internet-connection.json Normal file
View File

@ -0,0 +1,560 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"limit": 100,
"name": "Annotations & Alerts",
"showIn": 0,
"type": "dashboard"
}
]
},
"editable": true,
"gnetId": null,
"graphTooltip": 0,
"links": [],
"panels": [
{
"cacheTimeout": null,
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [
{
"id": 0,
"op": "=",
"text": "N/A",
"type": 1,
"value": "null"
}
],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "#299c46",
"value": null
},
{
"color": "blue",
"value": 100
}
]
},
"unit": "bps"
},
"overrides": []
},
"gridPos": {
"h": 9,
"w": 6,
"x": 0,
"y": 0
},
"id": 8,
"interval": null,
"links": [],
"maxDataPoints": 50000,
"options": {
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"mean"
],
"fields": "",
"values": false
},
"showThresholdLabels": false,
"showThresholdMarkers": true,
"text": {}
},
"pluginVersion": "7.5.6",
"targets": [
{
"expr": "speedtest_download_bits_per_second{}",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "",
"refId": "A"
}
],
"title": "Download",
"type": "gauge"
},
{
"cacheTimeout": null,
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [
{
"id": 0,
"op": "=",
"text": "N/A",
"type": 1,
"value": "null"
}
],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "blue",
"value": 20
}
]
},
"unit": "bps"
},
"overrides": []
},
"gridPos": {
"h": 9,
"w": 6,
"x": 6,
"y": 0
},
"id": 10,
"interval": null,
"links": [],
"maxDataPoints": 50000,
"options": {
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"mean"
],
"fields": "",
"values": false
},
"showThresholdLabels": false,
"showThresholdMarkers": true,
"text": {}
},
"pluginVersion": "7.5.6",
"targets": [
{
"expr": "speedtest_upload_bits_per_second{}",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "",
"refId": "A"
}
],
"title": "Upload",
"type": "gauge"
},
{
"cacheTimeout": null,
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"decimals": 2,
"mappings": [
{
"id": 0,
"op": "=",
"text": "N/A",
"type": 1,
"value": "null"
}
],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "ms"
},
"overrides": []
},
"gridPos": {
"h": 9,
"w": 12,
"x": 12,
"y": 0
},
"id": 12,
"interval": null,
"links": [],
"maxDataPoints": 50000,
"options": {
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"mean"
],
"fields": "",
"values": false
},
"showThresholdLabels": false,
"showThresholdMarkers": true,
"text": {}
},
"pluginVersion": "7.5.6",
"targets": [
{
"expr": "speedtest_ping_latency_milliseconds",
"format": "time_series",
"instant": false,
"interval": "",
"intervalFactor": 1,
"legendFormat": "",
"refId": "A"
}
],
"title": "Speedtest Ping",
"type": "gauge"
},
{
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 50,
"gradientMode": "none",
"hideFrom": {
"graph": false,
"legend": false,
"tooltip": false
},
"lineInterpolation": "linear",
"lineWidth": 0,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": true
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "bps"
},
"overrides": []
},
"gridPos": {
"h": 9,
"w": 12,
"x": 0,
"y": 9
},
"id": 6,
"links": [],
"maxDataPoints": 50000,
"options": {
"graph": {},
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "right"
},
"tooltipOptions": {
"mode": "multi"
}
},
"pluginVersion": "7.4.5",
"repeat": null,
"targets": [
{
"expr": "speedtest_download_bits_per_second{}",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "Download",
"refId": "A"
},
{
"expr": "speedtest_upload_bits_per_second{}",
"hide": false,
"interval": "",
"legendFormat": "Upload",
"refId": "B"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Speedtest Graph",
"type": "timeseries"
},
{
"cards": {
"cardHSpacing": 2,
"cardMinWidth": 5,
"cardRound": null,
"cardVSpacing": 2
},
"color": {
"cardColor": "#b4ff00",
"colorScale": "sqrt",
"colorScheme": "interpolateGnYlRd",
"defaultColor": "#757575",
"exponent": 0.5,
"max": 1,
"min": 0,
"mode": "discrete",
"thresholds": [
{
"$$hashKey": "object:68",
"color": "#37872D",
"tooltip": "Up",
"value": "1"
},
{
"$$hashKey": "object:72",
"color": "#C4162A",
"tooltip": "Down",
"value": "0"
}
]
},
"datasource": "prometheus",
"fieldConfig": {
"defaults": {},
"overrides": []
},
"gridPos": {
"h": 9,
"w": 12,
"x": 12,
"y": 9
},
"highlightCards": true,
"id": 14,
"legend": {
"show": false
},
"links": [],
"nullPointMode": "as empty",
"pageSize": 15,
"seriesFilterIndex": -1,
"statusmap": {
"ConfigVersion": "v1"
},
"targets": [
{
"expr": "probe_success",
"format": "time_series",
"interval": "",
"intervalFactor": 1,
"legendFormat": "{{instance}}",
"refId": "A"
}
],
"title": "Uptime",
"tooltip": {
"extraInfo": "",
"freezeOnClick": true,
"items": [],
"show": true,
"showExtraInfo": false,
"showItems": false
},
"type": "flant-statusmap-panel",
"useMax": true,
"usingPagination": false,
"xAxis": {
"show": true
},
"yAxis": {
"maxWidth": -1,
"minWidth": -1,
"show": true
},
"yAxisSort": "metrics",
"yLabel": {
"delimiter": "",
"labelTemplate": "",
"usingSplitLabel": false
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "prometheus",
"fieldConfig": {
"defaults": {},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 24,
"x": 0,
"y": 18
},
"hiddenSeries": false,
"id": 4,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"maxPerRow": 12,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.5.6",
"pointradius": 5,
"points": false,
"renderer": "flot",
"repeat": "host",
"repeatDirection": "h",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(probe_http_duration_seconds) by (instance)",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "{{instance}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "HTTP Request Duration",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
}
],
"refresh": "5m",
"schemaVersion": 27,
"style": "dark",
"tags": [
"speedtest",
"ping"
],
"templating": {
"list": []
},
"time": {
"from": "now-7d",
"to": "now"
},
"timepicker": {
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
]
},
"timezone": "browser",
"title": "Internet connection",
"uid": "o9mIe_Aik",
"version": 12
}

271
grafana/provisioning/dashboards/power-consumption.json Normal file
View File

@ -0,0 +1,271 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"gnetId": null,
"graphTooltip": 0,
"id": 2,
"links": [],
"panels": [
{
"datasource": "prometheus",
"description": "",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"custom": {},
"mappings": [],
"max": 500,
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "#EAB839",
"value": 150
},
{
"color": "red",
"value": 300
}
]
},
"unit": "watt"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 0
},
"id": 6,
"options": {
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showThresholdLabels": false,
"showThresholdMarkers": true,
"text": {}
},
"pluginVersion": "7.4.5",
"targets": [
{
"expr": "power",
"interval": "",
"legendFormat": "Watts",
"queryType": "randomWalk",
"refId": "A"
}
],
"title": "Power Consumption (Latest)",
"type": "gauge"
},
{
"datasource": "prometheus",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"custom": {},
"mappings": [],
"max": 500,
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "#EAB839",
"value": 150
},
{
"color": "red",
"value": 300
}
]
},
"unit": "watt"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 0
},
"id": 4,
"options": {
"reduceOptions": {
"calcs": [
"mean"
],
"fields": "",
"values": false
},
"showThresholdLabels": false,
"showThresholdMarkers": true,
"text": {}
},
"pluginVersion": "7.4.5",
"targets": [
{
"expr": "power",
"interval": "",
"legendFormat": "Watts",
"queryType": "randomWalk",
"refId": "A"
}
],
"title": "Power Consumption (Average)",
"type": "gauge"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "prometheus",
"description": "",
"fieldConfig": {
"defaults": {
"custom": {},
"unit": "watt"
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 8
},
"hiddenSeries": false,
"id": 2,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": false,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 1,
"points": true,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "power",
"interval": "",
"legendFormat": "Watts",
"queryType": "randomWalk",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Power Consumption",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:169",
"decimals": null,
"format": "watt",
"label": null,
"logBase": 1,
"max": null,
"min": "0",
"show": true
},
{
"$$hashKey": "object:170",
"decimals": null,
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
}
],
"refresh": "5m",
"schemaVersion": 27,
"style": "dark",
"tags": [
"power"
],
"templating": {
"list": []
},
"time": {
"from": "now-6h",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Power consumption",
"uid": "i_aeo-uMz",
"version": 3
}

1960
grafana/provisioning/dashboards/starlink-overview.json Normal file
View File

File diff suppressed because one or more lines are too long

BIN
images/dashboard.jpeg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 381 KiB

BIN
images/dashboard.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 196 KiB

3
prometheus/pinghosts.yaml
View File

@ -1,8 +1,7 @@
- targets: # url;humanname;routing;switch
- http://www.google.com/;google.com;external;internetbox
#- https://github.com/;github.com;external;internetbox
- https://github.com/;github.com;external;internetbox
- https://www.apple.com/;apple.com;external;internetbox
- https://www.fibracat.cat/;fibracat.cat;external;internetbox
# - http://192.168.1.1;internetbox;local;internetbox
# - http://192.168.1.42;blackdata;local;orbirouter-attic
# - http://192.168.1.101;orbirouter;local;orbirouter

27
prometheus/prometheus.yml
View File

@ -11,7 +11,7 @@ rule_files:
scrape_configs:
- job_name: 'prometheus'
scrape_interval: 60s
scrape_interval: 5s
static_configs:
- targets: ['localhost:9090']
@ -22,9 +22,28 @@ scrape_configs:
static_configs:
- targets: ['speedtest:9798']
# This job config only works with my internet-pi configuration currently,
# if you also have the shelly plug enabled there. It will just error out
# by default if you don't have a Shelly Plug on your network. Someday I'll
# work on making this more configurable...
- job_name: 'shelly-plug'
metrics_path: /metrics
scrape_interval: 1m
static_configs:
- targets: ['172.17.0.1:9924']
# This job config only works with my internet-pi configuration currently,
# if you also have starlink enabled there. It will just error out by default
# if you don't have Starlink. Someday I'll work on making this more
# configurable...
- job_name: 'starlink'
metrics_path: /metrics
static_configs:
- targets: ['172.17.0.1:9817']
- job_name: 'ping'
metrics_path: /probe
scrape_interval: 60s
scrape_interval: 5s
params:
module: [http_2xx] # Look for a HTTP 200 response.
file_sd_configs:
@ -55,7 +74,3 @@ scrape_configs:
- job_name: 'nodeexp'
static_configs:
- targets: ['nodeexp:9100']
- job_name: 'python-portschecker'
static_configs:
- targets: ['portschecker:9116']

10
pyPrometheusPortChecker/config/config.yml
View File

@ -1,10 +0,0 @@
service_port: 9116
scrape_frequency: 3600
host_to_check: yourdomain.is
ports_tocheck:
- 25
- 80
- 443
- 32400
- 22
- 51413