grammar and clarifications
parent
c16c5721ed
commit
47fa6657c8
|
|
@ -5,29 +5,32 @@ date: 2023-08-27T15:37:00+03:00
|
|||
|
||||
Just before this summer I [laid out my roadmap]({{< ref
|
||||
"log/2023/roadmap-summer-2023.md" >}}) for, well, the summer. Here is a quick
|
||||
retrospective. Weird order, because it is somewhat chronological.
|
||||
retrospective. Weird order, because it is somewhat chronological. This
|
||||
post does not offer background for the project, so the previous post is a
|
||||
prerequisite reading.
|
||||
|
||||
Project 2: my take on universal headers
|
||||
---------------------------------------
|
||||
|
||||
During [SYCL 2023][sycl2023] I was told that [Johny Marler][marler] expressed
|
||||
explicit interest in the project. I have much higher hopes in him than myself
|
||||
-- at which point I decided there are better things to do and shelved
|
||||
universal-headers. Non-ironically, I am no longer working on compiling lots of
|
||||
C++ code, so this area has less ROI for the potentially invested time.
|
||||
--- at which point I decided to shelve universal-headers. Non-ironically, I am
|
||||
no longer working on compiling lots of C++ code, so this area has less ROI than
|
||||
it did back in the day at Uber.
|
||||
|
||||
Project 1: city limits
|
||||
----------------------
|
||||
|
||||
Showing any spatial result requires a background map. In the spirit of
|
||||
protomaps, I looked into how to host the base maps myself. That was quite a
|
||||
rabbit hole with twists and turns, designed to misguide the non-initiated.
|
||||
Showing any spatial result requires a map in the background (e.g. OpenStreetMap
|
||||
tiles). In the spirit of protomaps, I looked into how to host the base layers
|
||||
myself. That was quite a rabbit hole, turns out, designed to misguide the
|
||||
non-initiated.
|
||||
|
||||
Since maps are simple pictures (PNG squares) with a bit of JavaScript to
|
||||
download and display them, I wanted to "keep things simple" and serve the files
|
||||
with a plain web server. Which meant pre-generating many tiles from the
|
||||
OpenStreetMap data. Turns out, this results in [5 million tiles][calculator]
|
||||
for Lithuania:
|
||||
Since online maps are PNG squares with a bit of JavaScript for download and
|
||||
display, I wanted to "keep things simple" and serve the files with a simple web
|
||||
server which also hosts my blog. To do it, I pre-generated many tiles from the
|
||||
OpenStreetMap data. Turns out, this results in [5 million files][calculator] to
|
||||
cover Lithuania:
|
||||
|
||||
{{<img src="_/2023/tile-calculator.png"
|
||||
alt="screenshot from geofabrik.com"
|
||||
|
|
@ -35,56 +38,47 @@ for Lithuania:
|
|||
hint="graph"
|
||||
>}}
|
||||
|
||||
Generating 5 million tiles takes at least a day on my small server (I gave up
|
||||
after this time having decided to seek for a different approach). To generate
|
||||
tiles one still needs the usual suspects — PostGIS and friends. So
|
||||
pre-generating Lithuania with a complex process and many dependencies and then
|
||||
serving them via a web server does not really save on "simplicity".
|
||||
Generating 5 million tiles takes at least a day on my small server. To generate
|
||||
tiles one still needs the usual suspects — PostGIS and more dependencies.
|
||||
Pre-generating Lithuania does not really save on "simplicity", so I decided to
|
||||
look for another approach.
|
||||
|
||||
After I discarded the pre-generated tiles idea, I looked at how to serve them
|
||||
on the fly. This is how most of the world does it, so this is the beaten path.
|
||||
The usual OpenStreetMap stack consists of:
|
||||
How about serving them on the fly? This is how most of the world does it,
|
||||
should be straightforward. The usual OpenStreetMap stack consists of:
|
||||
|
||||
1. PostGIS with all the data, and the tools to keep it up to date.
|
||||
2. [mapnik][mapnik], a map renderer. A library written in C++.
|
||||
3. `renderd`: a daemon that renders map tiles using mapnik.
|
||||
2. [mapnik][mapnik], a map renderer library written in C++.
|
||||
3. `renderd`: a daemon that renders map tiles. Uses mapnik.
|
||||
4. [`mod_tile`][mod_tile]: An Apache module that connects to `renderd` and
|
||||
spits the tiles out in HTTP.
|
||||
|
||||
I have no interest in maintaining Apache for just the base maps, so `mod_tile`
|
||||
would need to be replaced. [`go_tile`][go_tile] is a good candidate. During my
|
||||
earlier pre-generation phases I found that `image/webp` are about half the size
|
||||
of PNG. As a result, `mod_tile` [can now render
|
||||
`image/webp`](https://github.com/openstreetmap/mod_tile/pull/318) and `go_tile`
|
||||
[can use it](https://github.com/nielsole/go_tile/pull/13).
|
||||
of PNG. I could not resist looking into `image/webp`, so now `mod_tile` [can
|
||||
now render `image/webp`](https://github.com/openstreetmap/mod_tile/pull/318)
|
||||
and `go_tile` [can use it](https://github.com/nielsole/go_tile/pull/13).
|
||||
|
||||
During the middle of all this I understood how my personal stack is unfit for
|
||||
the purpose of ad-hoc projects like this: if I put out something for the world
|
||||
to see, I want it to keep working for years. Serving the base tiles is just
|
||||
part of the problem, but already includes many more moving parts than I would
|
||||
like to maintain on my crumbling servers. So I decided to shelve this mapping
|
||||
project and spend some attention spans to "my infrastructure".
|
||||
At this point I realized my personal stack is unfit for this project: if I put
|
||||
out something for the world to see, I want it to keep working for years.
|
||||
Serving the base tiles is just part of the problem, but already includes many
|
||||
more moving parts than I would like to maintain on my crumbling servers. So I
|
||||
shelved the "city boundaries" and devoted my attention to "personal
|
||||
infrastructure".
|
||||
|
||||
Project 3a: home lab "infrastructure"
|
||||
------------------------------------
|
||||
|
||||
As alluded in the previous section, the "home infrastructure" was not in a
|
||||
shape I would be happy to maintain long-term. If I start a project that I can
|
||||
show off, it needs to work for a long time with no or minimal maintenance.
|
||||
|
||||
At the beginning of Summer I ran two Debian servers that needed to be upgraded.
|
||||
They have been configured by two thousand lines of Ansible YAMLs. I felt that
|
||||
writing the yamls were a necessary pain, so grinded it. It was so painful to
|
||||
configure stuff in Ansible that I wrote my own http server instead of using
|
||||
nginx; I wrote my own DNS server instead of using bind or nsd. And I was ready
|
||||
to write even more software from scratch, myself, just because it was so hard
|
||||
to configure out-of-the-box components with Ansible.
|
||||
At the beginning of Summer I had been running two Debian servers waiting for an
|
||||
upgrade. They have been configured by two thousand lines of Ansible YAMLs,
|
||||
which was painful. Painful enough, so I wrote my own DNS and HTTP servers just
|
||||
to avoid configuration with yaml.
|
||||
|
||||
In an unrelated conversation with my ex-colleague and good friend Ken Micklas,
|
||||
he suggested taking a more serious look into NixOS. I spent most of the
|
||||
summer's "computer time" understanding and dabbing at Nix. Boy it was a great
|
||||
decision. Time will tell, of course, for how long it was a maintainable one,
|
||||
but it shines pretty bright colors now.
|
||||
he suggested taking a more serious look into NixOS. It immediately clicked,
|
||||
and, as a result, I spent most of the screen time dabbing at Nix and migrating
|
||||
my servers. Time will tell if it was a great long-term decision, but it looks
|
||||
pretty good now.
|
||||
|
||||
As of writing, I have the following on a small [Odroid H2+][oh2] computer in my
|
||||
closet:
|
||||
|
|
@ -110,7 +104,7 @@ closet:
|
|||
verification][config-nsd-acme].
|
||||
* All on an encrypted root file system in ZFS, with nightly backups to
|
||||
rsync.net. The encrypted file system allowed me the liberty to add private
|
||||
data on the server: family photos, chat histories, later -- host a password
|
||||
data on the server: family photos, chat histories, later --- host a password
|
||||
manager for my family.
|
||||
|
||||
DNS server is a reason why I run another server. The second server is an
|
||||
|
|
@ -122,21 +116,16 @@ AArch64 virtual machine in Hetzner for €3.98/month which serves two purposes:
|
|||
each other in case either of them reboots (hopefully not both at the same
|
||||
time).
|
||||
|
||||
Why encrypted root partition at home? Because I don't want my family photos and
|
||||
personal documents unencrypted.
|
||||
|
||||
Project 3b: home and vacation
|
||||
-----------------------------
|
||||
|
||||
Originally I planned to take some time off and do all those projects. But after
|
||||
a couple of weeks it turned out that I am unfit for such a regime: the full day
|
||||
goes by and I do not feel like I have achieved more than I would normally have
|
||||
if I were employed.
|
||||
a couple of weeks it turned out that I am unfit for such schedule. Day ends and
|
||||
I do not feel like I have achieved more than I would normally have if I were
|
||||
employed. So at mid-August I started actively interviewing. It is ongoing now;
|
||||
I expect to have a job again sooner than I originally thought.
|
||||
|
||||
So at mid-August I started actively interviewing. It is ongoing now; I expect
|
||||
to have a job again sooner than I originally thought.
|
||||
|
||||
I will keep you posted! Next -- Matrix.
|
||||
I will keep you posted! Next --- Matrix.
|
||||
|
||||
[calculator]: https://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=20.602031,53.844653,26.82,56.45
|
||||
[mod_tile]: https://github.com/openstreetmap/mod_tile
|
||||
|
|
|
|||
Loading…
Reference in New Issue