152 lines
8.5 KiB
Markdown
152 lines
8.5 KiB
Markdown
---
|
|
title: "End of Summer 2023"
|
|
date: 2023-08-27T15:37:00+03:00
|
|
---
|
|
|
|
Just before this summer I [laid out my roadmap]({{< ref
|
|
"log/2023/roadmap-summer-2023.md" >}}) for, well, the summer. Here is a quick
|
|
retrospective. Weird order, because it is somewhat chronological. This
|
|
post does not offer background for the project, so the previous post is a
|
|
prerequisite reading.
|
|
|
|
Project 2: my take on universal headers
|
|
---------------------------------------
|
|
|
|
During [SYCL 2023][sycl2023] I was told that [Johny Marler][marler] expressed
|
|
explicit interest in the project. I have much higher hopes in him than myself
|
|
--- at which point I decided to shelve universal-headers. Non-ironically, I am
|
|
no longer working on compiling lots of C++ code, so this area has less ROI than
|
|
it did back in the day at Uber.
|
|
|
|
Project 1: city limits
|
|
----------------------
|
|
|
|
Showing any spatial result requires a map in the background (e.g. OpenStreetMap
|
|
tiles). In the spirit of protomaps, I looked into how to host the base layers
|
|
myself. That was quite a rabbit hole, turns out, designed to misguide the
|
|
non-initiated.
|
|
|
|
Since online maps are PNG squares with a bit of JavaScript for download and
|
|
display, I wanted to "keep things simple" and serve the files with a simple web
|
|
server which also hosts my blog. To do it, I pre-generated many tiles from the
|
|
OpenStreetMap data. Turns out, this results in [5 million files][calculator] to
|
|
cover Lithuania:
|
|
|
|
{{<img src="_/2023/tile-calculator.png"
|
|
alt="screenshot from geofabrik.com"
|
|
caption="Pre-generating Lithuania to zoom level 17 would output 5 million tiles."
|
|
hint="graph"
|
|
>}}
|
|
|
|
Generating 5 million tiles takes at least a day on my small server. To generate
|
|
tiles one still needs the usual suspects — PostGIS and more dependencies.
|
|
Pre-generating Lithuania does not really save on "simplicity", so I decided to
|
|
look for another approach.
|
|
|
|
How about serving them on the fly? This is how most of the world does it,
|
|
should be straightforward. The usual OpenStreetMap stack consists of:
|
|
|
|
1. PostGIS with all the data, and the tools to keep it up to date.
|
|
2. [mapnik][mapnik], a map renderer library written in C++.
|
|
3. `renderd`: a daemon that renders map tiles. Uses mapnik.
|
|
4. [`mod_tile`][mod_tile]: An Apache module that connects to `renderd` and
|
|
spits the tiles out in HTTP.
|
|
|
|
I have no interest in maintaining Apache for just the base maps, so `mod_tile`
|
|
would need to be replaced. [`go_tile`][go_tile] is a good candidate. During my
|
|
earlier pre-generation phases I found that `image/webp` are about half the size
|
|
of PNG. I could not resist looking into `image/webp`, so now `mod_tile` [can
|
|
now render `image/webp`](https://github.com/openstreetmap/mod_tile/pull/318)
|
|
and `go_tile` [can use it](https://github.com/nielsole/go_tile/pull/13).
|
|
|
|
At this point I realized my personal stack is unfit for this project: if I put
|
|
out something for the world to see, I want it to keep working for years.
|
|
Serving the base tiles is just part of the problem, but already includes many
|
|
more moving parts than I would like to maintain on my crumbling servers. So I
|
|
shelved the "city boundaries" and devoted my attention to "personal
|
|
infrastructure".
|
|
|
|
Project 3a: home lab "infrastructure"
|
|
------------------------------------
|
|
|
|
At the beginning of Summer I had been running two Debian servers waiting for an
|
|
upgrade. They have been configured by two thousand lines of Ansible YAMLs,
|
|
which was painful. Painful enough, so I wrote my own DNS and HTTP servers just
|
|
to avoid configuration with yaml.
|
|
|
|
In an unrelated conversation with my ex-colleague and good friend Ken Micklas,
|
|
he suggested taking a more serious look into NixOS. It immediately clicked,
|
|
and, as a result, I spent most of the screen time dabbing at Nix and migrating
|
|
my servers. Time will tell if it was a great long-term decision, but it looks
|
|
pretty good now.
|
|
|
|
As of writing, I have the following on a small [Odroid H2+][oh2] computer in my
|
|
closet:
|
|
|
|
* This web server ([jakstys.lt][config-jakstys.lt]).
|
|
* [Syncthing][config-syncthing] to synchronize documents, photos and podcasts
|
|
between my laptop, phone and server.
|
|
* [My code hosting instance][config-gitea], which is also a "single sign-on"
|
|
provider. I authenticate to Headscale, Grafana and Gitea using it. I still
|
|
have separate passwords for IRC and Matrix. IRC has no SSO for being IRC, and
|
|
for Matrix it's ["on the roadmap"][oidc-matrix] for a while now.
|
|
* [Home VPN for all my devices][config-headscale].
|
|
* [Prometheus + Grafana][config-grafana], accessible only via the personal VPN.
|
|
* [IRC bouncer][config-soju], so I can visit `#zig` on libera.chat.
|
|
* [Matrix server][config-synapse], so I can still visit NixOS channels. Blog
|
|
post about my declining usage of Matrix is coming soon.
|
|
* [Automatic updates][config-deployerbot] with automatic rollbacks when
|
|
upgrades fail.
|
|
* DNS server. Here is the [zone config][config-dns] and the [server
|
|
config][config-nsd]. This allows me to not use the "free" DNS providers (I
|
|
have used one before, but had to move after a day-long outage). Having my own
|
|
DNS server allows me to have letsencrypt certificates with [DNS
|
|
verification][config-nsd-acme].
|
|
* All on an encrypted root file system in ZFS, with nightly backups to
|
|
rsync.net. The encrypted file system allowed me the liberty to add private
|
|
data on the server: family photos, chat histories, later --- host a password
|
|
manager for my family.
|
|
|
|
DNS server is a reason why I run another server. The second server is an
|
|
AArch64 virtual machine in Hetzner for €3.98/month which serves two purposes:
|
|
|
|
* The [DNS server][config-nsd-fra1].
|
|
* Remote [unlocking of the home machine's root
|
|
partition][config-zfsunlock-fra1]. The servers keep trying to ping and unlock
|
|
each other in case either of them reboots (hopefully not both at the same
|
|
time).
|
|
|
|
Project 3b: home and vacation
|
|
-----------------------------
|
|
|
|
Originally I planned to take some time off and do all those projects. But after
|
|
a couple of weeks it turned out that I am unfit for such schedule. Day ends and
|
|
I do not feel like I have achieved more than I would normally have if I were
|
|
employed. So at mid-August I started actively interviewing. It is ongoing now;
|
|
I expect to have a job again sooner than I originally thought.
|
|
|
|
I will keep you posted! Next --- Matrix.
|
|
|
|
[calculator]: https://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=20.602031,53.844653,26.82,56.45
|
|
[mod_tile]: https://github.com/openstreetmap/mod_tile
|
|
[go_tile]: https://github.com/nielsole/go_tile
|
|
[mapnik]: https://mapnik.org/
|
|
[sycl2023]: https://softwareyoucanlove.ca/
|
|
[marler]: https://github.com/marler8997/
|
|
[oh2]: https://ameridroid.com/products/odroid-h2
|
|
[config-headscale]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/headscale/default.nix
|
|
[config-jakstys.lt]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L189-L223
|
|
[config-grafana]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L180-L185
|
|
[config-soju]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L322-L332
|
|
[config-synapse]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/matrix-synapse/default.nix
|
|
[config-deployerbot]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/deployerbot/default.nix#L61-L68
|
|
[config-gitea]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/gitea/default.nix#L26-L68
|
|
[oidc-matrix]: https://areweoidcyet.com/
|
|
[config-dns]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/data.nix#L87-L118
|
|
[config-nsd]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L314-L320
|
|
[config-nsd-fra1]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/fra1-a/configuration.nix#L78-L84
|
|
[config-zfsunlock-fra1]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/fra1-a/configuration.nix#L61-L73
|
|
[config-nsd-acme]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L109-L119
|
|
[config-syncthing]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/syncthing/default.nix#L35-L39
|
|
|