1
Fork 0

grammar and clarifications

This commit is contained in:
Motiejus Jakštys 2023-08-28 00:29:16 +03:00
parent c16c5721ed
commit 47fa6657c8
1 changed files with 46 additions and 57 deletions

View File

@ -5,29 +5,32 @@ date: 2023-08-27T15:37:00+03:00
Just before this summer I [laid out my roadmap]({{< ref
"log/2023/roadmap-summer-2023.md" >}}) for, well, the summer. Here is a quick
retrospective. Weird order, because it is somewhat chronological.
retrospective. Weird order, because it is somewhat chronological. This
post does not offer background for the project, so the previous post is a
prerequisite reading.
Project 2: my take on universal headers
---------------------------------------
During [SYCL 2023][sycl2023] I was told that [Johny Marler][marler] expressed
explicit interest in the project. I have much higher hopes in him than myself
-- at which point I decided there are better things to do and shelved
universal-headers. Non-ironically, I am no longer working on compiling lots of
C++ code, so this area has less ROI for the potentially invested time.
--- at which point I decided to shelve universal-headers. Non-ironically, I am
no longer working on compiling lots of C++ code, so this area has less ROI than
it did back in the day at Uber.
Project 1: city limits
----------------------
Showing any spatial result requires a background map. In the spirit of
protomaps, I looked into how to host the base maps myself. That was quite a
rabbit hole with twists and turns, designed to misguide the non-initiated.
Showing any spatial result requires a map in the background (e.g. OpenStreetMap
tiles). In the spirit of protomaps, I looked into how to host the base layers
myself. That was quite a rabbit hole, turns out, designed to misguide the
non-initiated.
Since maps are simple pictures (PNG squares) with a bit of JavaScript to
download and display them, I wanted to "keep things simple" and serve the files
with a plain web server. Which meant pre-generating many tiles from the
OpenStreetMap data. Turns out, this results in [5 million tiles][calculator]
for Lithuania:
Since online maps are PNG squares with a bit of JavaScript for download and
display, I wanted to "keep things simple" and serve the files with a simple web
server which also hosts my blog. To do it, I pre-generated many tiles from the
OpenStreetMap data. Turns out, this results in [5 million files][calculator] to
cover Lithuania:
{{<img src="_/2023/tile-calculator.png"
alt="screenshot from geofabrik.com"
@ -35,56 +38,47 @@ for Lithuania:
hint="graph"
>}}
Generating 5 million tiles takes at least a day on my small server (I gave up
after this time having decided to seek for a different approach). To generate
tiles one still needs the usual suspects — PostGIS and friends. So
pre-generating Lithuania with a complex process and many dependencies and then
serving them via a web server does not really save on "simplicity".
Generating 5 million tiles takes at least a day on my small server. To generate
tiles one still needs the usual suspects — PostGIS and more dependencies.
Pre-generating Lithuania does not really save on "simplicity", so I decided to
look for another approach.
After I discarded the pre-generated tiles idea, I looked at how to serve them
on the fly. This is how most of the world does it, so this is the beaten path.
The usual OpenStreetMap stack consists of:
How about serving them on the fly? This is how most of the world does it,
should be straightforward. The usual OpenStreetMap stack consists of:
1. PostGIS with all the data, and the tools to keep it up to date.
2. [mapnik][mapnik], a map renderer. A library written in C++.
3. `renderd`: a daemon that renders map tiles using mapnik.
2. [mapnik][mapnik], a map renderer library written in C++.
3. `renderd`: a daemon that renders map tiles. Uses mapnik.
4. [`mod_tile`][mod_tile]: An Apache module that connects to `renderd` and
spits the tiles out in HTTP.
I have no interest in maintaining Apache for just the base maps, so `mod_tile`
would need to be replaced. [`go_tile`][go_tile] is a good candidate. During my
earlier pre-generation phases I found that `image/webp` are about half the size
of PNG. As a result, `mod_tile` [can now render
`image/webp`](https://github.com/openstreetmap/mod_tile/pull/318) and `go_tile`
[can use it](https://github.com/nielsole/go_tile/pull/13).
of PNG. I could not resist looking into `image/webp`, so now `mod_tile` [can
now render `image/webp`](https://github.com/openstreetmap/mod_tile/pull/318)
and `go_tile` [can use it](https://github.com/nielsole/go_tile/pull/13).
During the middle of all this I understood how my personal stack is unfit for
the purpose of ad-hoc projects like this: if I put out something for the world
to see, I want it to keep working for years. Serving the base tiles is just
part of the problem, but already includes many more moving parts than I would
like to maintain on my crumbling servers. So I decided to shelve this mapping
project and spend some attention spans to "my infrastructure".
At this point I realized my personal stack is unfit for this project: if I put
out something for the world to see, I want it to keep working for years.
Serving the base tiles is just part of the problem, but already includes many
more moving parts than I would like to maintain on my crumbling servers. So I
shelved the "city boundaries" and devoted my attention to "personal
infrastructure".
Project 3a: home lab "infrastructure"
------------------------------------
As alluded in the previous section, the "home infrastructure" was not in a
shape I would be happy to maintain long-term. If I start a project that I can
show off, it needs to work for a long time with no or minimal maintenance.
At the beginning of Summer I ran two Debian servers that needed to be upgraded.
They have been configured by two thousand lines of Ansible YAMLs. I felt that
writing the yamls were a necessary pain, so grinded it. It was so painful to
configure stuff in Ansible that I wrote my own http server instead of using
nginx; I wrote my own DNS server instead of using bind or nsd. And I was ready
to write even more software from scratch, myself, just because it was so hard
to configure out-of-the-box components with Ansible.
At the beginning of Summer I had been running two Debian servers waiting for an
upgrade. They have been configured by two thousand lines of Ansible YAMLs,
which was painful. Painful enough, so I wrote my own DNS and HTTP servers just
to avoid configuration with yaml.
In an unrelated conversation with my ex-colleague and good friend Ken Micklas,
he suggested taking a more serious look into NixOS. I spent most of the
summer's "computer time" understanding and dabbing at Nix. Boy it was a great
decision. Time will tell, of course, for how long it was a maintainable one,
but it shines pretty bright colors now.
he suggested taking a more serious look into NixOS. It immediately clicked,
and, as a result, I spent most of the screen time dabbing at Nix and migrating
my servers. Time will tell if it was a great long-term decision, but it looks
pretty good now.
As of writing, I have the following on a small [Odroid H2+][oh2] computer in my
closet:
@ -110,7 +104,7 @@ closet:
verification][config-nsd-acme].
* All on an encrypted root file system in ZFS, with nightly backups to
rsync.net. The encrypted file system allowed me the liberty to add private
data on the server: family photos, chat histories, later -- host a password
data on the server: family photos, chat histories, later --- host a password
manager for my family.
DNS server is a reason why I run another server. The second server is an
@ -122,21 +116,16 @@ AArch64 virtual machine in Hetzner for €3.98/month which serves two purposes:
each other in case either of them reboots (hopefully not both at the same
time).
Why encrypted root partition at home? Because I don't want my family photos and
personal documents unencrypted.
Project 3b: home and vacation
-----------------------------
Originally I planned to take some time off and do all those projects. But after
a couple of weeks it turned out that I am unfit for such a regime: the full day
goes by and I do not feel like I have achieved more than I would normally have
if I were employed.
a couple of weeks it turned out that I am unfit for such schedule. Day ends and
I do not feel like I have achieved more than I would normally have if I were
employed. So at mid-August I started actively interviewing. It is ongoing now;
I expect to have a job again sooner than I originally thought.
So at mid-August I started actively interviewing. It is ongoing now; I expect
to have a job again sooner than I originally thought.
I will keep you posted! Next -- Matrix.
I will keep you posted! Next --- Matrix.
[calculator]: https://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=20.602031,53.844653,26.82,56.45
[mod_tile]: https://github.com/openstreetmap/mod_tile