motiejus
/
jakstys.lt
1
Fork 0
Code Releases Activity

matrix

main
Motiejus Jakštys 2023-08-27 17:12:37 +03:00
parent 117f8cfb79
commit 2c8eb267c9
2 changed files with 162 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
assets/_/2023/tile-calculator.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.3 MiB

162
content/log/2023/end-of-summer.md Normal file
View File

@ -0,0 +1,162 @@
---
title: "End of Summer 2023"
date: 2023-08-27T15:37:00+03:00
---
Just before this summer I [laid out my roadmap]({{< ref
"log/2023/roadmap-summer-2023.md" >}}) for, well, the summer. Here is a quick
retrospective. Weird order, because it is somewhat chronological.
Project 2: my take on universal headers
---------------------------------------
During [SYCL 2023][sycl2023] I was told that [Johny Marler][marler] expressed
explicit interest in the project. I have much higher hopes in him than myself
-- at which point I decided there are better things to do and shelved
universal-headers. Non-ironically, I am no longer working on compiling lots of
C++ code, so this area has less ROI for the potentially invested time.
Project 1: city limits
----------------------
Showing any spatial result requires a background map. In the spirit of
protomaps, I looked into how to host the base maps myself. That was quite a
rabbit hole with twists and turns, designed to misguide the non-initiated.
Since maps are simple pictures (PNG squares) with a bit of JavaScript to
download and display them, I wanted to "keep things simple" and serve the files
with a plain web server. Which meant pre-generating many tiles from the
OpenStreetMap data. Turns out, this results in [5 million tiles][calculator]
for Lithuania:
{{<img src="_/2023/tile-calculator.png"
alt="screenshot from geofabrik.com"
caption="Pre-generating Lithuania to zoom level 17 would output 5 million tiles."
hint="graph"
>}}
Generating 5 million tiles takes at least a day on my small server (I gave up
after this time having decided to seek for a different approach). To generate
tiles one still needs the usual suspects — PostGIS and friends. So
pre-generating Lithuania with a complex process and many dependencies and then
serving them via a web server does not really save on "simplicity".
After I discarded the pre-generated tiles idea, I looked at how to serve them
on the fly. This is how most of the world does it, so this is the beaten path.
The usual OpenStreetMap stack consists of:
1. PostGIS with all the data, and the tools to keep it up to date.
2. [mapnik][mapnik], a map renderer. A library written in C++.
3. `renderd`: a daemon that renders map tiles using mapnik.
4. [`mod_tile`][mod_tile]: An Apache module that connects to `renderd` and
spits the tiles out in HTTP.
I have no interest in maintaining Apache for just the base maps, so `mod_tile`
would need to be replaced. [`go_tile`][go_tile] is a good candidate. During my
earlier pre-generation phases I found that `image/webp` are about half the size
of PNG. As a result, `mod_tile` [can now render
`image/webp`](https://github.com/openstreetmap/mod_tile/pull/318) and `go_tile`
[can use it](https://github.com/nielsole/go_tile/pull/13).
During the middle of all this I understood how my personal stack is unfit for
the purpose of ad-hoc projects like this: if I put out something for the world
to see, I want it to keep working for years. Serving the base tiles is just
part of the problem, but already includes many more moving parts than I would
like to maintain on my crumbling servers. So I decided to shelve this mapping
project and spend some attention spans to "my infrastructure".
Project 3a: home lab "infrastructure"
------------------------------------
As alluded in the previous section, the "home infrastructure" was not in a
shape I would be happy to maintain long-term. If I start a project that I can
show off, it needs to work for a long time with no or minimal maintenance.
At the beginning of Summer I ran two Debian servers that needed to be upgraded.
They have been configured by two thousand lines of Ansible YAMLs. I felt that
writing the yamls were a necessary pain, so grinded it. It was so painful to
configure stuff in Ansible that I wrote my own http server instead of using
nginx; I wrote my own DNS server instead of using bind or nsd. And I was ready
to write even more software from scratch, myself, just because it was so hard
to configure out-of-the-box components with Ansible.
In an unrelated conversation with my ex-colleague and good friend Ken Micklas,
he suggested taking a more serious look into NixOS. I spent most of the
summer's "computer time" understanding and dabbing at Nix. Boy it was a great
decision. Time will tell, of course, for how long it was a maintainable one,
but it shines pretty bright colors now.
As of writing, I have the following on a small [Odroid H2+][oh2] computer in my
closet:
* This web server ([jakstys.lt][config-jakstys.lt]).
* [Syncthing][config-syncthing] to synchronize documents, photos and podcasts
between my laptop, phone and server.
* [My code hosting instance][config-gitea], which is also a "single sign-on"
provider. I authenticate to Headscale, Grafana and Gitea using it. I still
have separate passwords for IRC and Matrix. IRC has no SSO for being IRC, and
for Matrix it's ["on the roadmap"][oidc-matrix] for a while now.
* [Home VPN for all my devices][config-headscale].
* [Prometheus + Grafana][config-grafana], accessible only via the personal VPN.
* [IRC bouncer][config-soju], so I can visit `#zig` on libera.chat.
* [Matrix server][config-synapse], so I can still visit NixOS channels. Blog
post about my declining usage of Matrix is coming soon.
* [Automatic updates][config-deployerbot] with automatic rollbacks when
upgrades fail.
* DNS server. Here is the [zone config][config-dns] and the [server
config][config-nsd]. This allows me to not use the "free" DNS providers (I
have used one before, but had to move after a day-long outage). Having my own
DNS server allows me to have letsencrypt certificates with [DNS
verification][config-nsd-acme].
* All on an encrypted root file system in ZFS, with nightly backups to
rsync.net. The encrypted file system allowed me the liberty to add private
data on the server: family photos, chat histories, later -- host a password
manager for my family.
DNS server is a reason why I run another server. The second server is an
AArch64 virtual machine in Hetzner for €3.98/month which serves two purposes:
1. The [DNS server][config-nsd-fra1].
1. Remote [unlocking of the home machine's root
partition][config-zfsunlock-fra1]. The servers keep trying to ping and
unlock each other in case either of them reboots (hopefully not both at the
same time).
Why encrypted root partition at home? Because I don't want my family photos and
personal documents unencrypted.
Project 3b: home and vacation
-----------------------------
Originally I planned to take some time off and do all those projects. But after
a couple of weeks it turned out that I am unfit for such a regime: the full day
goes by and I do not feel like I have achieved more than I would normally have
if I were employed.
So at mid-August I started actively interviewing. It is ongoing now; I expect
to have a job again sooner than I originally thought.
I will keep you posted! Next -- Matrix.
[calculator]: https://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=20.602031,53.844653,26.82,56.45
[mod_tile]: https://github.com/openstreetmap/mod_tile
[go_tile]: https://github.com/nielsole/go_tile
[mapnik]: https://mapnik.org/
[sycl2023]: https://softwareyoucanlove.ca/
[marler]: https://github.com/marler8997/
[oh2]: https://ameridroid.com/products/odroid-h2
[config-headscale]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/headscale/default.nix
[config-jakstys.lt]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L189-L223
[config-grafana]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L180-L185
[config-soju]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L322-L332
[config-synapse]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/matrix-synapse/default.nix
[config-deployerbot]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/deployerbot/default.nix#L61-L68
[config-gitea]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/gitea/default.nix#L26-L68
[oidc-matrix]: https://areweoidcyet.com/
[config-dns]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/data.nix#L87-L118
[config-nsd]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L314-L320
[config-nsd-fra1]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/fra1-a/configuration.nix#L78-L84
[config-zfsunlock-fra1]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/fra1-a/configuration.nix#L61-L73
[config-nsd-acme]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/hosts/vno1-oh2/configuration.nix#L109-L119
[config-syncthing]: https://git.jakstys.lt/motiejus/config/src/commit/97ef691743cc1b7014055aa47ab70ba7ee529b5f/modules/services/syncthing/default.nix#L35-L39