From 47fa6657c8c8c822c94623cd9fa43bff2d9425c0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Motiejus=20Jak=C5=A1tys?= Date: Mon, 28 Aug 2023 00:29:16 +0300 Subject: [PATCH] grammar and clarifications --- content/log/2023/end-of-summer.md | 103 +++++++++++++----------------- 1 file changed, 46 insertions(+), 57 deletions(-) diff --git a/content/log/2023/end-of-summer.md b/content/log/2023/end-of-summer.md index a5079e3..484e51e 100644 --- a/content/log/2023/end-of-summer.md +++ b/content/log/2023/end-of-summer.md @@ -5,29 +5,32 @@ date: 2023-08-27T15:37:00+03:00 Just before this summer I [laid out my roadmap]({{< ref "log/2023/roadmap-summer-2023.md" >}}) for, well, the summer. Here is a quick -retrospective. Weird order, because it is somewhat chronological. +retrospective. Weird order, because it is somewhat chronological. This +post does not offer background for the project, so the previous post is a +prerequisite reading. Project 2: my take on universal headers --------------------------------------- During [SYCL 2023][sycl2023] I was told that [Johny Marler][marler] expressed explicit interest in the project. I have much higher hopes in him than myself --- at which point I decided there are better things to do and shelved -universal-headers. Non-ironically, I am no longer working on compiling lots of -C++ code, so this area has less ROI for the potentially invested time. +--- at which point I decided to shelve universal-headers. Non-ironically, I am +no longer working on compiling lots of C++ code, so this area has less ROI than +it did back in the day at Uber. Project 1: city limits ---------------------- -Showing any spatial result requires a background map. In the spirit of -protomaps, I looked into how to host the base maps myself. That was quite a -rabbit hole with twists and turns, designed to misguide the non-initiated. +Showing any spatial result requires a map in the background (e.g. OpenStreetMap +tiles). In the spirit of protomaps, I looked into how to host the base layers +myself. That was quite a rabbit hole, turns out, designed to misguide the +non-initiated. -Since maps are simple pictures (PNG squares) with a bit of JavaScript to -download and display them, I wanted to "keep things simple" and serve the files -with a plain web server. Which meant pre-generating many tiles from the -OpenStreetMap data. Turns out, this results in [5 million tiles][calculator] -for Lithuania: +Since online maps are PNG squares with a bit of JavaScript for download and +display, I wanted to "keep things simple" and serve the files with a simple web +server which also hosts my blog. To do it, I pre-generated many tiles from the +OpenStreetMap data. Turns out, this results in [5 million files][calculator] to +cover Lithuania: {{screenshot from geofabrik.com}} -Generating 5 million tiles takes at least a day on my small server (I gave up -after this time having decided to seek for a different approach). To generate -tiles one still needs the usual suspects — PostGIS and friends. So -pre-generating Lithuania with a complex process and many dependencies and then -serving them via a web server does not really save on "simplicity". +Generating 5 million tiles takes at least a day on my small server. To generate +tiles one still needs the usual suspects — PostGIS and more dependencies. +Pre-generating Lithuania does not really save on "simplicity", so I decided to +look for another approach. -After I discarded the pre-generated tiles idea, I looked at how to serve them -on the fly. This is how most of the world does it, so this is the beaten path. -The usual OpenStreetMap stack consists of: +How about serving them on the fly? This is how most of the world does it, +should be straightforward. The usual OpenStreetMap stack consists of: 1. PostGIS with all the data, and the tools to keep it up to date. -2. [mapnik][mapnik], a map renderer. A library written in C++. -3. `renderd`: a daemon that renders map tiles using mapnik. +2. [mapnik][mapnik], a map renderer library written in C++. +3. `renderd`: a daemon that renders map tiles. Uses mapnik. 4. [`mod_tile`][mod_tile]: An Apache module that connects to `renderd` and spits the tiles out in HTTP. I have no interest in maintaining Apache for just the base maps, so `mod_tile` would need to be replaced. [`go_tile`][go_tile] is a good candidate. During my earlier pre-generation phases I found that `image/webp` are about half the size -of PNG. As a result, `mod_tile` [can now render -`image/webp`](https://github.com/openstreetmap/mod_tile/pull/318) and `go_tile` -[can use it](https://github.com/nielsole/go_tile/pull/13). +of PNG. I could not resist looking into `image/webp`, so now `mod_tile` [can +now render `image/webp`](https://github.com/openstreetmap/mod_tile/pull/318) +and `go_tile` [can use it](https://github.com/nielsole/go_tile/pull/13). -During the middle of all this I understood how my personal stack is unfit for -the purpose of ad-hoc projects like this: if I put out something for the world -to see, I want it to keep working for years. Serving the base tiles is just -part of the problem, but already includes many more moving parts than I would -like to maintain on my crumbling servers. So I decided to shelve this mapping -project and spend some attention spans to "my infrastructure". +At this point I realized my personal stack is unfit for this project: if I put +out something for the world to see, I want it to keep working for years. +Serving the base tiles is just part of the problem, but already includes many +more moving parts than I would like to maintain on my crumbling servers. So I +shelved the "city boundaries" and devoted my attention to "personal +infrastructure". Project 3a: home lab "infrastructure" ------------------------------------ -As alluded in the previous section, the "home infrastructure" was not in a -shape I would be happy to maintain long-term. If I start a project that I can -show off, it needs to work for a long time with no or minimal maintenance. - -At the beginning of Summer I ran two Debian servers that needed to be upgraded. -They have been configured by two thousand lines of Ansible YAMLs. I felt that -writing the yamls were a necessary pain, so grinded it. It was so painful to -configure stuff in Ansible that I wrote my own http server instead of using -nginx; I wrote my own DNS server instead of using bind or nsd. And I was ready -to write even more software from scratch, myself, just because it was so hard -to configure out-of-the-box components with Ansible. +At the beginning of Summer I had been running two Debian servers waiting for an +upgrade. They have been configured by two thousand lines of Ansible YAMLs, +which was painful. Painful enough, so I wrote my own DNS and HTTP servers just +to avoid configuration with yaml. In an unrelated conversation with my ex-colleague and good friend Ken Micklas, -he suggested taking a more serious look into NixOS. I spent most of the -summer's "computer time" understanding and dabbing at Nix. Boy it was a great -decision. Time will tell, of course, for how long it was a maintainable one, -but it shines pretty bright colors now. +he suggested taking a more serious look into NixOS. It immediately clicked, +and, as a result, I spent most of the screen time dabbing at Nix and migrating +my servers. Time will tell if it was a great long-term decision, but it looks +pretty good now. As of writing, I have the following on a small [Odroid H2+][oh2] computer in my closet: @@ -110,7 +104,7 @@ closet: verification][config-nsd-acme]. * All on an encrypted root file system in ZFS, with nightly backups to rsync.net. The encrypted file system allowed me the liberty to add private - data on the server: family photos, chat histories, later -- host a password + data on the server: family photos, chat histories, later --- host a password manager for my family. DNS server is a reason why I run another server. The second server is an @@ -122,21 +116,16 @@ AArch64 virtual machine in Hetzner for €3.98/month which serves two purposes: each other in case either of them reboots (hopefully not both at the same time). -Why encrypted root partition at home? Because I don't want my family photos and -personal documents unencrypted. - Project 3b: home and vacation ----------------------------- Originally I planned to take some time off and do all those projects. But after -a couple of weeks it turned out that I am unfit for such a regime: the full day -goes by and I do not feel like I have achieved more than I would normally have -if I were employed. +a couple of weeks it turned out that I am unfit for such schedule. Day ends and +I do not feel like I have achieved more than I would normally have if I were +employed. So at mid-August I started actively interviewing. It is ongoing now; +I expect to have a job again sooner than I originally thought. -So at mid-August I started actively interviewing. It is ongoing now; I expect -to have a job again sooner than I originally thought. - -I will keep you posted! Next -- Matrix. +I will keep you posted! Next --- Matrix. [calculator]: https://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=20.602031,53.844653,26.82,56.45 [mod_tile]: https://github.com/openstreetmap/mod_tile