jgrogan/personal-site
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add some self-hosting notes and a 2024 review
Some checks failed
/ test (push) Has been cancelled
Details

Browse source
This commit is contained in:
jgrogan 2024-12-18 10:03:51 +00:00
parent 6d11e20c6b
commit 0dfab69e72
4 changed files with 315 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

101
src/articles/self_hosting.md Normal file
View file

@ -0,0 +1,101 @@
# Self Hosting
This article is intended to be an ever-evolving guide to setting up a self-hosting solution, put together as I try out and learn various steps myself.
## Location ##
I want to host with my own hardware, mostly so I have control over costs but general control and independence reasons are appealing also.
I don't have a static IP and don't really want the fuss of getting and maintaining one, nor making my home IP publically visible. For this reason I've opted to run a small VPS on an external hosting provider to act as a public gateway to locally hosted content via a reverse proxy. I also want to run a VPN on this VPS, so I can access services only visible to my home network while away from it. For the external provider I went with Hetzner for reasons of price, reputation and that they are based in Europe. I wasn't too happy with the extent of personal data they collect during registration but understand there are positives to it also, such as IP ranges not being (as) overrun with crap.
## Hardware ##
I had an old Raspberry Pi laying around but felt it wouldn't have enough juice to handle the bunch of services I had planned for it so grabbed a Raspberry Pi 5 with 8 GB RAM. This seems to have been a wise choice when importing my media libraries at least.
I had a couple of old SSDs too - one 256 GB and one 2 TB, so hooked them up via PCIe to USB connectors.
The Pi is running on WiFi at the moment, although I could attach it to Ethernet over the house wiring if there is a performance issue.
## Operating System ##
For now, I've gone with the default Debian image for the Raspberry Pi - although it has plenty of bloat that I don't need. Hopefully in future I can get Guix running on there, which I'm currently using on my laptop.
## Core Services ##
I am planning to set up:
### DNS: Pi-Hole ###
https://docs.pi-hole.net/ Which I want to use for internal DHCP/DNS management, helpful for the likes of NextCloud - however the ad blocking functionality is certainly appealing too.
Followed option 'Alternative 1' here:
```
git clone --depth 1 https://github.com/pi-hole/pi-hole.git Pi-hole
cd "Pi-hole/automated install/"
sudo bash basic-install.sh
```
Before starting off you will need to force the machine to get a static IP, this is through `/etc/network/interfaces`.
Once set up for my router I needed the pi-hole to act as both DNS and DHCP servers. Did this by forcing the router DHCP server to only assign IP, a fixed one to the pi and then enabling DHCP serving on the Pi.
### VPN: Wireguard ###
I want to use Wireguard to allow access to services on my home network that are not intended to be public. It will also allow serving of services that are meant to be public via the VPS and reverse proxy.
https://www.digitalocean.com/community/tutorials/how-to-set-up-wireguard-on-ubuntu-20-04
### Backup ###
Considering rsync or BorgBackup
### Updates ###
Manual or watchtower
### Monitoring ###
Undecided
## Services ##
### Credentials ###
Vaultwarden
### Email ###
Custom self host for receive with, custom and gmail for sending
### Photos ###
Immich
### Music ###
Navidrome
### Documents ###
Nextcloud
### Calendar, Contacts, Notes ###
Nextcloud
### Code ###
Forgejo
### Personal Site ###
Proxied through VPN and reverse proxy
### Social ###
Self host Mastodon or GoToSocial

212
src/blogs/technical/end_of_year_recap_2024.md Normal file
View file

@ -0,0 +1,212 @@
This is a recap of some technical projects I've picked up, started or finished in 2024 and things I'd like to pick up in 2025.
# Relevant Background #
I'm a software developer with an interest in free software, open science, minimalism and yak shaving. I've been using Linux for about 15 years and finally rid myself of all things Windows some time last year. I've primarily worked with C++ and Python and think they are, just ok.
I had mostly used Ubuntu for personal computing and a little CentOS/RedHat at work, but started looking around for other distros when they introduced Snaps and killed CentOS 8 respectively, cementing my feelings that corporate distros should only be used for corporate things. My distro search included brief spells with:
* Debian - found packages too out of date
* Fedora - found dnf slow on my clunky old laptop
* Gentoo - never got it fully set up correctly
before settling on Arch some time in 2023.
I've primarily worked with GUI tools including IDEs and heavy mouse use, dropping to the terminal only when needed.
# 2024 Recap #
## Operating Systems ##
I had been running Arch for much of 2023 using Wayland and `suckless` tools as much as possible, e.g. `dwl`. The system was stable but I stuggled with the keyboard focused flow and sometimes needing a tool in a hurry - e.g. joining a wifi network on the road and trying to remember the terminal commands to do so.
I switched to Debian Sid early in the year - with the lofty goal of being able to contribute useful defect reports and maybe being able to get involved in some package maintenance. I switched back to GNOME and mousing about at the same time, which was a bit of a relief although floating windows did start to grate a bit.
I learned about Guix and along with being interested in learning Emacs decided to give the Guix System a shot, around March some time. I found the initial setup a bit tricky - since I needed some non-free kernel bits, but (presumably with some thanks to previous Gentoo experience) eventually pulled a system together. It has been rock solid since and I think my distro hopping days are over for now.
The big appeals of Guix to me are:
* having control over the assembly of the system and being able to build something minimal
* having a clean language to configure the system (Guile Scheme)
* its free software and community focus
* use of an alternative to SystemD - I think competition and having choice are healthy and I'm not running an enterprise on my laptop
* maybe some day using GNU Hurd - did I mention the yak shaving?
With the switch to Guix I switched to Sway as a window-manager, rather than `dwl`. Interestingly it is more minimal than I had thought feature-wise, properly setting up `sway-lock` and `sway-idle` are still on my todo list. This allowed me to remove a lot of GNOME and GDM things from my Guix setup (with David Wilson's awesome blog, videos and repo doing most of the heavy lifting). There is still a long way to go in terms of my understanding the lower level Linux userspace before chopping more off my Guix config, but that is what 2025 is for.
## IDEs ##
Switching IDE was definitely my biggest and most time-consuming technical project of 2024. I haven't been happy with my IDE choices - basically ever. The Eclipse IDE was one of the first I used for Python and C++. It is free software and I'm greatful to the developers for building it, but I found every single aspect of it painful to use, from crashes to lockups and runaway resource use while indexing to a painful plugin management system, and did I mention the Java?
After many years of using Eclipse I used Visual Studio while working on a Windows-centric product for several years. It was adequate but I don't like the Microsoft C++ development environment for several reasons. Once I no longer had to write code for Windows I switched to VS Code on all platforms. VS Code is an amazing IDE, performance is great and tooling and plugins 'just work'. In my current role I write tutorials and best practice documentation for other developers and the general public. I'm really not a fan of Microsoft and although there is 'free' VS Codium with some of the nasties stripped out I'm wary of the now dominance of VS Code and the possibility of a web-browser like co-option of standards by a small number of large corporations. so, it didn't feel good to me to make documentation with this IDE as the prominent option.
This left me with the choice of Emacs and the Vi ecosystem. Since I was trying out Guix at the same time and was interested in the idea of LISP I decided to give Emacs a try. The experience is worthy of a dedicated post - but needless to say that transition was hugely time consuming and challening - but I'm out the other side now and things are looking good. Once again I'm finally satisfied I've found a platform I can stick with for a long time.
## Self Hosting and De-Googling ##
2024 has been a good year for first steps at self-hosting and degoogling. Some of the things I set up have been:
* Moving from Github to Codeberg and then primarily a self-hosted Forgejo instance
* Moving my personal site from Github pages and Python pelican generator to a custom generator and self-hosted server
* Making a self-hosted dynamic website for my wedding with CSS and Vanilla JS
* Moving all of my credentials to Bitwarden
* Moving my mail from Google to Migadu
* Dropping my Spotify and using Bandcamp, with self-hosted Navidrome for listening
* Archiving my photos via Immich self-hosting
* Setting up a Raspberry Pi for local self-hosting
* Running pi-hole for DNS blackholing and DHCP
## Languages ##
2024 hasn't been very active for picking up new languages or language features. I mostly got back up to speed with recent developments in Typescript (and Angular) and Python. For Python the main things I picked up were the packaging system changes (e.g. Pyproject.toml), use of linters and the typing system (which I'm enjoying).
I did learn a bit about RPM packaging at the start of the year, which is possibly forgotten by now.
One change I did make was trying out more functional approaches to programming. I learned programming via OOP and have been using that everywhere. For some Python projects I've tried a more functional approach with immutable data classes and functions without side-effects and it has been pretty satisfying - I'm going to go further down this road in 2025 with language choices.
## Side Projects and Open Source Contributions ##
2024 has been a mixed to dissappoiting year on the side projects front. For several (at least 5) years now I've been trying to build a library of 'things from scratch', e.g. a :
* compiler
* build system (e.g. make)
* pdf, png, midi, wav, svg, xml, json, yaml readers and writers
* 3d renderer
* GUI system
* static site generator (including markdown, html parsers)
* web server and client
I had originally written them in modern C++, but it didn't feel 'from scratch' enough. So I wrote replacements for STL containers and memory management - basically seeing if I could run something without the C++ standard library. If you are shouting 'why?!' right now - know that I am too.
The original goal of this 'from scratch' library was to build tooling for myself to use - since I mostly end up hitting rough edges or things I don't like about other tools. That and of course the 'for fun' or yak shaving element. However with Emacs and increased comfort in keyboard-driven workflows I need this tooling less now. I hope to use pieces of this library in 2025, while learning some of the other languages, particularly C, Rust and Guile Scheme.
The side project I did get completed was a website for my wedding. This was a 'from scratch' HTML/CSS/Vanilla JS frontend with a Cherrypy backend. It came out looking fairly ok, although buidling it at the same time as planning a wedding was probably unwise.
I am still yet to contribute to a (non-work) collaborative open source project - not good. Hopefully that will change in 2025.
## Analysis and Theory ##
I haven't spent much time brushing up on analytical or theoretical skills this year - I've had a brief interaction with computational mechanics again, namely crystal plasticity theory but I can see more theoretical research on the horizon in 2025.
## Sustainability ##
I grew my own vegetables in my small back garden this year including setting up a chunky box-planter. This was very sucsessful and should hopefully be back next year.
I cycle to work most days but it has been a tough year for bike maintenance. I have a weird racer bike and seemingly need to do most maintenance myself - anything that can break on the bike has this year. Learning how to fix each issue has been slow.
# 2025 Plans #
And so, the lofty plans for 2025...
## Operating Systems ##
* Write up how low-level Linux ecosystem services work and fit together (e.g. PAM, elogind, DBus, Wayland seats) in the context of Guix, so I learn about them
* Handle power managment on my laptop - including Sway Idle, at the moment I don't know how to suspend or check if it is in low power mode
* Add a little icon to sway bar for charging
* Handle sound management on laptop - currently I have no keyboard control over sound levels
* Learn about and implement Guix Home
* Further customize and simplify my Guix config (yes, nice and vague)
* Learn about Nix and Nix Home for my work Mac
## IDEs ##
* Try to not depend on treemacs so much - try dired only or similar
* Add email handling to Emacs config
* Learn more Emacs and Vim keyboard shortcuts - particularly document navigation and search
* Learn basic regex
* Learn and document how Emacs packaging works
* Learn magit properly
* Find a comfortable way to work with Emacs shell
* Find a comfortable way to browse web with Emacs
* Learn to use a LISP REPL in Emacs
* Learn Python and C debugging and profiling in Emacs
* Come up with a comfortable Emacs and Vim config for remote terminal sessions.
* Troubleshoot slow initial file search in Emacs on Mac
* Link Emacs forge support with forgejo
## Self Hosting ##
* Set up a backup solution
* Expose locally hosted services via Wireguard and cloud instance
* Set up Nextcloud for documents, calendar and contacts
* Set up own mail server for receive
* Move forgejo to local hosting and set up a Foregejo runner
* Set up service monitoring
* Set up local network monitoring and altering
* Network my USB printer via an RPI
## Languages ##
This is the area where I have most FOMO in terms of what to spend my time on (FWIW I have zero FOMO on not learning AI APIs or prompt engineering). There is such a rich environment of modern languages and development ecosystems to chose from and it seems there is plenty of scope for exciting developments.
At a meta level - I would like to learn how compilers work and programming language theory and design, but I'm not sure 2025 will allow me to go too deep there.
### C
Of course, I 'known' C - I've written snippets of code and tutorials in it and have lots of C++ experience, but I don't 'know know' C. I've never written enough of it to have to deal with complexities, edge cases or to have thought about code style. I'd like to learn more of it since it is still foundational to low-level Linux utilities - which I want to understand, modify and maybe contribute to. This year I'd like to write a full side-project or two in C.
### Rust ###
I like the idea of Rust and I'm tired of C++ and its package management disaster zone. Still, I'm wary of Rust's own approach to package management - seemingly following the 'pip' and 'npm' roads. I'm also wary of static linking and the complex build and compile ecosystem. While I do think it is possible in theory to write moderately safe C++ with suitable use of its modern pointers, I have sadly been exposed to large C++ projects in the real world.
So overall, the future is looking bright for Rust and all impressions I've read are that the language itself is pleasant to use. So, I'm on board. Hopefully I can get a project or two completed in Rust this year.
### Guile Scheme ###
I'm quite sure I want to learn some Lisp - I've read too much on how different, powerful and fun it is not to and would like to build up my functional programming skills. Since it is now powering my operating system and IDE, it would seem important to know a bit too.
From reading around Scheme seems a well regarded Lisp dialect to pick up, and given my 'free software' leanings and use of Guix then Guile Scheme seems a good implementation. So, hopefully some Guile Scheme projects are on the horizon too.
I'm curious about more 'powerful' dialects I've heard about too, like Racket and Clojure although they may be for another year - along with Haskel on the functional programming odysee.
## Side Projects and Open Source Contributions ##
### Personal Site ###
I'd like to get my personal site into better shape as a blog, including:
* supporting blog posts
* supporting RSS
* looking a bit nicer
Currently it is being built using Python - I'm thinking about using Scheme instead, possibly with Rust for Markdown parsing - which is related to another project of interest.
For blog posts, some I'd like to cover include:
* A review of the Supercomputing conference SC24
* A retrospective on moving from Vs Code to Emacs
* A review of my vegetable patch experience
I'd also like to start making some videos - so some OBs practice is on the cards.
### Open Source ###
I'd like to contribute to Forgejo and particularly efforts around federation. I think that is a really promising feature for Open Science (which I think has no business depending on Github) where labs or institutes host their own forges with rich linked data features that Github will never build.
I'd also like to contribute to Guix - I'm not sure how yet but maybe I can do something useful around infrastructure or testing.
### Creative ###
I'd like to get back making 3d renders and some electronic music - I'm not sure what form they'll take yet - maybe I can involve Scheme somehow.
### Analysis and Theory ###
Some areas I'd like to cover more:
* Continuum Mechanics, Crystal Plasticity Theory and Finite Element solvers - getting back up to speed
* Implementation of digitial quantum simulators - e.g. how to solve quantum circuits on classical computers
* How transformers work in Machine Learning - my ML knowledge pretty much stops in 2017
# And Beyond... #
What is the big plan or picture? Basically:
* having a minimal, contrarian computing setup that I understand well and have a good degree of control over. Something involving Guix, Hurd, Emacs etc.
* having a comfortable setup for creative activities: writing, 3D rendering, video creation and music creation
* having a low footprint hosting setup - both financial cost and resource use-wise
* keeping up-to-date with modern technology and being able to tell hype from genuinely interesting progress

0
src/blogs/technical/self_hosting.md Normal file
View file

4
src/index.md
View file

@ -12,10 +12,10 @@ I enjoy writing software and live in the West of Ireland. My main interests are:
This site includes a [technical](/blogs/technical) and [social](/blogs/social) blog, some info on previous and on-going [projects](/projects.html) and a [cv](/cv.html).
My personal software projects are [hosted here](https://git.jmsgrogan.com/). Many of my work projects are open source, you can find some through my [work profile](https://git.ichec.ie/jgrogan).
My personal software projects are [hosted here](https://git.jmsgrogan.com/jgrogan). Many of my work projects are open source, you can find some through my [work profile](https://git.ichec.ie/jgrogan).
I am on <a rel="me" href="https://mastodon.social/@compilz">Mastodon</a>, but not any other social media sites.
Unless otherwise specified all content on this site was created by me and can be re-used under the terms of the [CC BY-SA 4.0 license](https://creativecommons.org/licenses/by-sa/4.0/deed.en).
I can be contacted at: <br/> <img src="/static/contact.png" width=220px>
I can be contacted at: <br/> <img src="/static/contact.png" width=280px>