From 060c7de471b3c81d7f64f835f6b42140dacb5367 Mon Sep 17 00:00:00 2001 From: jgrogan Date: Sun, 9 Feb 2025 15:34:48 +0000 Subject: [PATCH] Add some self hosting notes --- src/articles/interesting_sites.md | 24 +++++++++++++ src/articles/linux_tools.md | 58 +++++++++++++++++++++++++++++++ src/articles/self_hosting.md | 38 +++++++++++--------- 3 files changed, 103 insertions(+), 17 deletions(-) create mode 100644 src/articles/linux_tools.md diff --git a/src/articles/interesting_sites.md b/src/articles/interesting_sites.md index dc0f66a..e4f44f6 100644 --- a/src/articles/interesting_sites.md +++ b/src/articles/interesting_sites.md @@ -80,6 +80,8 @@ This is an unstructured list of web-sites I find interesting. I keep them here r * [sockets](https://www.cs.rpi.edu/~moorthy/Courses/os98/Pgms/socket.html) * [http made easy](https://www.jmarshall.com/easy/http/) +* [Wireguard for private network](https://taggart-tech.com/wireguard/) +* [Sysadmin tools](https://mwl.io/nonfiction/tools) # Database # @@ -113,9 +115,16 @@ This is an unstructured list of web-sites I find interesting. I keep them here r * [Fuzzing GStreamer](https://github.blog/security/vulnerability-research/uncovering-gstreamer-secrets/) +# Maths # + +* [The Matrix Cookbook](https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf) +* [The Tensor Cookbook](https://tensorcookbook.com/) + # Machine Learning # * [MLE](https://rish-01.github.io/blog/posts/ml_estimation/) +* [Gradient Descent Overview](https://www.ruder.io/optimizing-gradient-descent/) +* [Generative Models Overview](https://nrehiew.github.io/blog/2024/) # Linked Data # @@ -125,6 +134,21 @@ This is an unstructured list of web-sites I find interesting. I keep them here r * [Flappy Bird C Clone](https://github.com/alxyng/flappybird) +# Operating Systems # + +* [Unix History](http://crn.hopto.org/unix/) + + +# Live and Minimalist Computing # + +* [Emacs like editor on steroids](https://albertzak.com/run-build-grow/) +* [Direnv and .envrc files](https://offby1.website/posts/uv-direnv-and-simple-envrc-files.html) + + +# Hardware # + +* [Making an analogue synth](https://www.peterzimon.com/hog/) + # Gardening # * [Planning a vegetable garden for beginners](https://www.youtube.com/watch?v=9R-utqpmwmE) diff --git a/src/articles/linux_tools.md b/src/articles/linux_tools.md new file mode 100644 index 0000000..337011f --- /dev/null +++ b/src/articles/linux_tools.md @@ -0,0 +1,58 @@ +# Logging and Monitoring # + +* rsyslog +* [syslog](https://en.wikipedia.org/wiki/Syslog): RFC5424 + +`/var/log/syslog` +`/var/log` + +* eBPF + +* Zabbix +* Influx +* Grafana +* Prometheus (+ Grafana + Loki as stack) +* timescaleDB +* AlertManager +* Loki +* Graphite +* Spiceworks +* Crowdsec +* Netdata +* NodeExtractor/NodeExporter +* ELK - Elasticsearch, Kibana, Logstash + + +https://grafana.com/blog/2016/01/05/logs-and-metrics-and-graphs-oh-my/ + + +Setting up Grafana: https://grafana.com/docs/grafana/latest/setup-grafana/installation/docker/ + +Setting up Prometheus: https://github.com/prometheus/prometheus + +Some things to measure: + +- apt status (for security/critical updates that haven't been run yet) +- reboot needed (presence of /var/run/reboot-required) +- fail2ban jail status (how many are in each of our defined jails) +- CPU usage +- MySQL active, long-running processes, number of queries +- iostat numbers +- disk space +- SSL cert expiration date +- domain expiration date +- reachability (ping, domain resolution, specific string in an HTTP request) +- Application-specific checks (WordPress, Drupal, CRM, etc) +- postfix queue size +* apt/yum/fwupd/... pending updates +* mailqueue length, root's mailbox size: this is an indicator for stuff going wrong silently +* pending reboot after kernel update +* certain kinds of log entries (block device read error, OOMkills, core dumps). +* network checksum errors, dropped packets, martians +* presence or non-presence of USB devices: desktops should have keyboard and mouse. servers usually shouldn't. usb storage is sometimes forbidden. + +## Further Reading ## + +* https://www.redhat.com/en/blog/log-aggregation-rsyslog + +# Auto updates diff --git a/src/articles/self_hosting.md b/src/articles/self_hosting.md index ab23d47..1dea92c 100644 --- a/src/articles/self_hosting.md +++ b/src/articles/self_hosting.md @@ -4,42 +4,46 @@ This article is intended to be an ever-evolving guide to setting up a self-hosti ## Location ## -I want to host with my own hardware, mostly so I have control over costs but general control and independence reasons are appealing also. +I want to host with my own hardware where feasible, mostly so I have control over costs but general control and independence reasons are appealing also. -I don't have a static IP and don't really want the fuss of getting and maintaining one, nor making my home IP publically visible. For this reason I've opted to run a small VPS on an external hosting provider to act as a public gateway to locally hosted content via a reverse proxy. I also want to run a VPN on this VPS, so I can access services only visible to my home network while away from it. For the external provider I went with Hetzner for reasons of price, reputation and that they are based in Europe. I wasn't too happy with the extent of personal data they collect during registration but understand there are positives to it also, such as IP ranges not being (as) overrun with crap. +I don't have a static IP and don't really want the fuss of getting one, nor making my home IP publically visible. For this reason I've opted to run a small VPS on an external hosting provider to act as a public gateway to locally hosted content. I also want to run a VPN on this VPS, so I can access services only visible to my home network while away from it. For the external provider I went with [Hetzner](https://www.hetzner.com/) for reasons of price, reputation and that they are based in Europe. I wasn't too happy with the extent of personal data they collect during registration but understand why they do it. ## Hardware ## -I had an old Raspberry Pi laying around but felt it wouldn't have enough juice to handle the bunch of services I had planned for it so grabbed a Raspberry Pi 5 with 8 GB RAM. This seems to have been a wise choice when importing my media libraries at least. +I have a hodge-podge of old hardware laying around and where possible am trying to use that over buying anything new. That said, my Raspberry Pi was ancient and I don't think it was up to the task of running my planned services, so I grabbed a Raspberry Pi 5 with 8 GB RAM. I had a couple of old SSDs - one 256 GB and one 2 TB, so hooked them up via PCIe to USB connectors. The Pi is running on WiFi at the moment, although I could attach it to Ethernet over the house wiring if there is a performance issue. -I had a couple of old SSDs too - one 256 GB and one 2 TB, so hooked them up via PCIe to USB connectors. +## Public Host ## -The Pi is running on WiFi at the moment, although I could attach it to Ethernet over the house wiring if there is a performance issue. +I'm running a Debian image on a small Hetzner VPS. This hosts public facing services, such as my personal website and Forgejo code repository. -## Operating System ## -For now, I've gone with the default Debian image for the Raspberry Pi - although it has plenty of bloat that I don't need. Hopefully in future I can get Guix running on there, which I'm currently using on my laptop. +## Home Network ## -## Core Services ## +My home network just consists of a Raspberry Pi based server for now - with access from various laptops, mobile clients and a Smart Tv. I don't have any short term plans to add to that. It isn't exposed to the public internet but is planned to be accessible by VPN. -I am planning to set up: +### Operating System ### -### DNS: Pi-Hole ### +For now, I've gone with the default Debian image for the Raspberry Pi. I wasn't too careful setting it up so have ended up with a full blown desktop installation. Hopefully in future I can eventually get [Guix](https://guix.gnu.org/) running on there, which I'm currently happily using on my laptop. -https://docs.pi-hole.net/ Which I want to use for internal DHCP/DNS management, helpful for the likes of NextCloud - however the ad blocking functionality is certainly appealing too. +### Core Services ### -Followed option 'Alternative 1' here: +These are the core services that support access, monitoring and security on my system. -``` +#### DNS #### + +I'm using [pi-hole](https://docs.pi-hole.net/) for ad blocking and am planning to also use it for internal DHCP/DNS management on my home network. + +For starters I set it up to act as a DHCP server. Before starting I needed to configure the pi to get a static IP, this is through `/etc/network/interfaces`. + +Then I followed option 'Alternative 1' on the pi-hole getting started docs: + +```sh git clone --depth 1 https://github.com/pi-hole/pi-hole.git Pi-hole cd "Pi-hole/automated install/" sudo bash basic-install.sh ``` -Before starting off you will need to force the machine to get a static IP, this is through `/etc/network/interfaces`. - -Once set up for my router I needed the pi-hole to act as both DNS and DHCP servers. Did this by forcing the router DHCP server to only assign IP, a fixed one to the pi and then enabling DHCP serving on the Pi. - +Finally I forced my router's DHCP server to only assign one IP, a fixed one to the pi and then enabled DHCP serving on the Pi. ### VPN: Wireguard ###