motiejus/jakstys.lt
1
Fork 0
Code Releases Activity

half-size img

Browse Source
This commit is contained in:
Motiejus Jakštys 2022-04-26 20:31:01 +03:00
parent d4838bb558
commit d311e0f66d
3 changed files with 130 additions and 81 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
assets/_/styles.scss
View File

@ -76,11 +76,15 @@ figure {
margin: 40px 0; margin: 40px 0;
} }
figure.right {
float: right;
}
img,figcaption { img,figcaption {
margin: 10px auto; margin: 10px auto;
padding: 0 10px; padding: 0 10px;
height: auto; height: auto;
display: block display: block;
} }
@media (max-width:600px) { @media (max-width:600px) {

201
content/log/2022/dependencies.md
View File

@ -11,62 +11,55 @@ draft: true
<!-- o_ --> <!-- o_ -->
TLDR: Modern programming languages make it very easy to add many dependencies. TLDR
That is nice for development, but a nightmare for maintenance. Unfortunately, ----
zig is following suit. I wish we could accept that adding dependencies does not
have to be trivial. If we accept that, thanks to ubiquity of git, we may have Modern programming languages make it very easy to add many dependencies. That
almost solved the dependency problem. is nice for development, but a nightmare for long-term maintenance.
Unfortunately, zig is following suit. I wish we could accept that adding
dependencies does not have to be trivial. If we accept that, thanks to ubiquity
of git, we may have almost solved the dependency problem: not only for zig, but
for everyone.
Adding dependencies Adding dependencies
------------------- -------------------
All of the programming languages I've used professionally, the names of which do not All of the programming languages I've used professionally, the names of which
start with "c"[^1], have package managers[^2], which make "dependency do not start with "c"[^1], have package managers[^2], which make "dependency
management" easy. These package managers will, as part of the project's build management" easy. These package managers will, as part of the project's build
process, download and build the dependencies, which makes adding and using process, download and build the dependencies. So there is virtually no
third-party dependencies easy. resistance to add dependencies when we need them.
Because C/C++ still does not have a universal package manager, not adding Because C/C++ still does not have a "universal" package manager, not adding
external dependencies to C/C++ is the path of least resistance; instead, one external dependencies to C/C++ is the path of least resistance; instead, one
relies on libraries already installed in the system. Therefore, there is a relies on libraries already installed in the system. There is a plethora of
plethora of dependency managers that will discover but not install tools that will discover system dependencies: autotools, cmake, pkg-config, and
dependencies: autotools, cmake, pkg-config, and others. As a result, C/C++ others. As a result, C/C++ projects I've participated in usually had 0-5
projects I've participated in usually had 0-5 non-system dependencies, whereas non-system dependencies, whereas non-C/C++ projects -- tens, hundreds, or
non-C/C++ projects -- tens, hundreds, or thousands[^3]. Having many system thousands[^3]. Having many system dependencies is painful for *every user* of
dependencies is painful for *every user* of the package (because they have to the package (because they have to make sure the libraries, and their correct
make sure the libraries, and their correct versions, are installed), so C/C++ versions, are installed), so C/C++ projects tend avoid having too many of them.
projects avoid having too many of them.
Not doing things that are easy to do requires discipline: brushing teeth,
limiting candy intake, not adding dependencies all over the place. If it is
easy to add dependencies and there is no discipline not doing so, the project
will gain a lot of dependency "weight" with time.
{{<img src="_/2022/brick-house.jpg"
alt="House made out of Duplo pieces"
caption="Just like this brick house, \"modern\" package managers are optimized for building, not maintenance. Photo mine, house by my sons."
hint="photo"
>}}
In Go and Python, a small number of dependencies is often a sign of care and In Go and Python, a small number of dependencies is often a sign of care and
quality. [mattn/go-sqlite3](https://github.com/mattn/go-sqlite3), quality. [mattn/go-sqlite3](https://github.com/mattn/go-sqlite3),
[uber/zap](https://github.com/uber-go/zap), [uber/zap](https://github.com/uber-go/zap),
[apenwarr/redo](https://github.com/apenwarr/redo) and [apenwarr/redo](https://github.com/apenwarr/redo) and
[django](https://djangoproject.com) are good examples. I've built and used [django](https://djangoproject.com) are good examples. I've built and used
these projects in a number of environments without issues. Conversely, projects these projects in a number of environments. Conversely, projects with many
with many dependencies often fail to build even in the environment they are dependencies, even when pinned, often fail to build even in the environment
developed and at and thus had received most testing (e.g. a specific they are developed at and thus had received most testing (e.g. a specific
OS+architecture, like `Ubuntu 16.04 x86_64`). It's even worse to do on a OS+architecture, like `Ubuntu 16.04 x86_64`). It's even worse if the
non-standard environment, no matter how trivially different (e.g. they would environment, no matter how trivially, is different from the one developer is
build on Ubuntu 16.04, but fail on Ubuntu 18.04), not to mention a different working at[^4]. Let's forget about a different OS or a different build system.
OS. This, obviously, leads to both user frustation, packagers' frustation, and Inability to build software, unsurprisingly, leads to user frustration,
developer long-term frustration and costs. packagers' frustration, and the developers asking themselves why have they
chosen a career in software instead of, say, farming.
The costs of just having dependencies are huge. I haven't done a survey and To recap, the costs of just having dependencies are huge. I haven't done a
have only my experience to base this on (read: "many anecdotes of me failing to survey and have only my experience to base this on (read: "many anecdotes of me
build stuff I wrote a decade ago"). But it is bad enough that I have a failing to build stuff I or others wrote a decade ago"). But it is bad enough
dependency checklist and am prepared to do the grunt work to save my future that I have a dependency checklist and am prepared to do the grunt work to save
self. Here is it: my future self. Here is it:
1. Does the dependency do what I want, does it work at all? 1. Does the dependency do what I want, does it work at all?
2. Is it well written? API surface, documentation, tests, error handling, error 2. Is it well written? API surface, documentation, tests, error handling, error
@ -76,70 +69,110 @@ self. Here is it:
4. It's system dependencies. 4. It's system dependencies.
5. It's transitive dependencies. 5. It's transitive dependencies.
Assuming a "programming-language-specific package manager that does what it's When working with a "programming-language-specific package manager that does
advertised to do", the path of least resistance, when it comes to this what it's advertised to do", the path of least resistance, when it comes to
checklist, is doing (1), and perhaps (2). Why bother with transitive this checklist, is doing (1), and perhaps (2). Why bother with transitive
dependencies or it's build complexity, if the package manager will take care of dependencies or it's build complexity, if the package manager takes care of it
it all anyway? all anyway?
Except it will only when you are adding it. Package manager will not help you Except package manager will only help during the initial development, when the
when the dependency disappears, its API changes, it stops doing what it has developer happily adds the package. It will work for a couple of days. Package
advertised and many other [problems][crash-of-leftpad]. manager will not help when the dependency disappears, its API changes, it stops
doing what it has advertised and many other [problems][crash-of-leftpad]. When
something breaks (and it inevitably will, unless it's SQLite), the work is on
the maintainer to fix it.
I am trying to do all 5. If a dependency is well written, but has more I am following my checklist. If a dependency is well written, but has more
transitive dependencies than I need and there is no good alternative, I will transitive dependencies than I need and there is no good alternative, I will
fork and trim it. My recent example is fork and trim it. My recent example is
[sql-migrate](https://github.com/motiejus/sql-migrate). [sql-migrate](https://github.com/motiejus/sql-migrate).
Not doing things that are easy to do requires discipline: brushing teeth,
limiting candy intake, not adding dependencies all over the place. If adding
dependencies is easy (and there is no established discipline of limiting them),
the project will tend to gain them; lots of them.
{{<img src="_/2022/brick-house.jpg"
alt="House made out of Duplo pieces"
caption="Just like this brick house, \"modern\" package managers are optimized for building, not maintenance. Photo mine, house by my sons."
hint="photo"
>}}
To sum up, the "modern" languages optimize for initial development experience, To sum up, the "modern" languages optimize for initial development experience,
not maintenance. And as [Corbet says][linux-rust], "We can't understand why not maintenance. And as [Corbet says][linux-rust], "We can't understand why
Kids These Days just don't want to live that way". Kids want to build, John, Kids These Days just don't want to live that way". Kids want to build, John,
not maintain. A 4-letter Danish corporation made a fortune by selling toys that not maintain. A 4-letter Danish corporation made a fortune by selling toys that
do not need to be maintained: they are designed to be disassembled and built do not need to be maintained: they are designed to be disassembled and built
anew. We are still kids. Growing up requires discipline, which is very hard, anew. We are still kids. Growing up and sticking to our own rules requires
when candy is cheap and package managers (and disks and network, which make all discipline.
of it possible) are as good as they are today.
If I may combine Corbet's views with mine: if we understand and audit our If I may combine Corbet's views with mine: if we understand and audit our
dependencies (all of them, including transitive ones), we will have less dependencies (all of them, including transitive ones), we will have less
dependencies and a more maintainable system. Win-win. dependencies and a more maintainable system. Win.
Which brings us to... Which brings us to git submodules and git-subtract.
git-subtrac git submodules and git-subtrac
----------- ------------------------------
[`git-subtrac`][git-subtrac] manages our git dependencies (in our git A quick primer on [git submodules][git-submodule], a prerequisite to understand
repository) just like "classic" git submodules, but all refs of the `git-subtrac`:
* A submodule is a pointer to a particular ref in a separate repository,
optionally checked out in our tree. For example, `deps/cmph` would contain
all the files from [cmph][cmph]. This means that once the repository is fully
set up (technically, the submodule is synced/updated), the build system
(Makefiles, build.zig or what have you) can use it just like a regular
directory.
* The pointer to the submodule in your repository is just a tuple: `(git URL,
sha1)`.
* When cloning a repository that has submodules, git will not clone the
submodules, it will just leave empty directories. We must pass `--recursive`
for git to clone everything. Which makes sense when submodules are external
and may not download at all.
Submodules were designed for adding external dependencies to a repository.
However, using them incorrectly is way too easy, and is not fun when happens. I
see at least these significant usability problems:
- It is too easy to commit unintended changes to submodule, causing misery to
others.
- By default submodule contents (i.e. code of your dependency) lives *outside
the repository*. This means that, with time, if dependency disappears, we
will not be able to compile our code. Gone.
Because of the many usability problems of submodules, very few people use it.
So [Avery Pennarun][apenwarr] (creator of [git-subtree][git-subtree], by the
way) created [`git-subtrac`][git-subtrac]. `git-subtrac` bundles our git
dependencies just like "classic" git submodules, but all refs of the
dependencies stay in the same repository. Wait, stop here. Repeat after me: _it dependencies stay in the same repository. Wait, stop here. Repeat after me: _it
is git submodules, but all refs stay in the same repository_. I also call it is git submodules, but all refs stay in the same repository_. I also call it
"good vendoring". Since all the deps are in our repo, no external force can "good vendoring". Since all the dependencies are in our repo, no external force
make our dependency unavailable, change without notice. And it will keep the can make our dependency unavailable. And it will keep the size of the
size of the repository in check, because it's all there when you pull it. repository in check, because it's all there when we pull it. [`git-subtrac`
fixes a few other submodule usability problems][apenwarr-subtrac] along the
way.
Because `git-subtrac` is a vendoring tool, not a package manager, it only Because `git-subtrac` is a vendoring tool, not a package manager, it only
vendors but does not help building packages. Therefore, with `git-subtrac` it vendors but does not help building packages. Therefore, with `git-subtrac` it
is harder to add and "make work" (build, test, add transitive deps) a is harder to add and "make work" (build, test, add transitive dependencies) a
dependency than with a language-specific package manager. Oh, what about the dependency than with a language-specific package manager.
transitive dependencies?
[`git-subtrac`][git-subtrac] does not deal with transitive dependencies. At `git-subtrac`, just like git and submodules, does not understand "semantic
least not directly. Or I am not aware of it. Ok, I haven't tried. versions". So we can't ask for "latest foo of version 1.2.X"; the developer
will need to figure out, and hardcode, *exactly* which versions to use. Also,
If we audit and thus understand our dependencies, we will be able to add the updating dependencies is not as easy as, say, in Gospeak, `go get -u ./...`;
transitive ones. So perhaps git-subtrac shouldn't care? git will need a bit more hand holding.
What about Zig? What about Zig?
--------------- ---------------
Zig will have a package manager ([ziglang/zig#943][943]). I am not not very Zig will have a package manager ([ziglang/zig#943][943]). I am not not very
enthusiastic about it; can we all use git-subtrac and be done with it?. A few enthusiastic about it; can we all use git-subtrac and be done with it? A few
weeks ago in a park in Milan my conversation with [Andrew weeks ago in a park in Milan my conversation with [Andrew
Kelley](https://andrewkelley.me/) was something like: Kelley](https://andrewkelley.me/) was something like:
- me: "git-subtrac yadda yadda yadda submodules but better yadda yadda yadda". - me: "git-subtrac yadda yadda yadda submodules but better yadda yadda yadda".
- Andrew: "If I clone a repository that uses it with no extra parameters, will - Andrew: "If I clone a repository that uses subtrac with no extra parameters,
it work as expected?" will it work as expected?"
- me: "No, you have to pass `--recursive`, so git will checkout submodules... - me: "No, you have to pass `--recursive`, so git will checkout submodules...
even if they are already fetched." even if they are already fetched."
- Andrew: "Then it's a piece-of-shit-approach." - Andrew: "Then it's a piece-of-shit-approach."
@ -148,15 +181,19 @@ Uh, I agree. People have not grown muscle memory to clone repositories with
`--recursive` flag and never will, so it's impossible to adopt git-subtrac `--recursive` flag and never will, so it's impossible to adopt git-subtrac
beyond well-controlled silos. Which is why we will have a beyond well-controlled silos. Which is why we will have a
yet-another-programming-language-specific-package-manager. Or at least my yet-another-programming-language-specific-package-manager. Or at least my
argument for using and advertising `git-subtrac` (and saving a lot of time for argument offering `git-subtrac` as Zig's package manager (thus saving a lot of
Zig folks, and a lot of inevitable misery for its users) stops right there. time for Zig folks, and a lot of inevitable misery for its users) stops right
there.
Zig has a rich standard library, therefore it does not need many dependencies
by design. Does it *really* need a package manager?
Conclusion Conclusion
---------- ----------
Can git check out submodules when they are in the same repository, so our When all contents of the submodules are in our repository, can git check out
conversation of reconsidering (or not having) a Zig package manager doesn't submodules too? That way, my and Andrew's conversation of reconsidering (or not
stop after 5 seconds? having) a Zig package manager will have a chance to not stop after 5 seconds.
[^1]: Alphabetically: Erlang, Go, Java, Javascript, PHP, Perl, Python. [^1]: Alphabetically: Erlang, Go, Java, Javascript, PHP, Perl, Python.
[^2]: Usually written in the same language. Zoo of package managers (sometimes [^2]: Usually written in the same language. Zoo of package managers (sometimes
@ -164,8 +201,14 @@ stop after 5 seconds?
in an on itself worth another blog post. in an on itself worth another blog post.
[^3]: `go.sum` of a project I am currently involved in clocks around 6k lines. [^3]: `go.sum` of a project I am currently involved in clocks around 6k lines.
This is quite a lot for Go, but still peanuts to Node.js. This is quite a lot for Go, but still peanuts to Node.js.
[^4]: For example, they would work on Ubuntu 16.04, but fail on Ubuntu 18.04.
[git-subtrac]: https://github.com/apenwarr/git-subtrac/ [git-subtrac]: https://github.com/apenwarr/git-subtrac/
[linux-rust]: https://lwn.net/SubscriberLink/889924/a733d6630e3b5115/ [linux-rust]: https://lwn.net/SubscriberLink/889924/a733d6630e3b5115/
[crash-of-leftpad]: https://drewdevault.com/2021/11/16/Cash-for-leftpad.html [crash-of-leftpad]: https://drewdevault.com/2021/11/16/Cash-for-leftpad.html
[943]: https://github.com/ziglang/zig/issues/943 [943]: https://github.com/ziglang/zig/issues/943
[git-submodule]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
[cmph]: http://cmph.sourceforge.net/
[git-subtree]: https://git.kernel.org/pub/scm/git/git.git/plain/contrib/subtree/git-subtree.txt
[apenwarr]: https://apenwarr.ca
[apenwarr-subtrac]: https://apenwarr.ca/log/20191109

4
layouts/shortcodes/img.html
View File

@ -15,6 +15,8 @@ care about is 700*4=2800px.
{{ $j1400 := $src.Resize "1400x" }} {{ $j1400 := $src.Resize "1400x" }}
{{ $j2800 := $src.Resize "2800x" }} {{ $j2800 := $src.Resize "2800x" }}
{{ $width := cond (eq (.Get "half") "true") "350px" "700px" }}
{{ with .Get "hint" }} {{ with .Get "hint" }}
{{ else }} {{ else }}
{{ errorf "missing value for param 'hint': %s" .Position }} {{ errorf "missing value for param 'hint': %s" .Position }}
@ -29,7 +31,7 @@ care about is 700*4=2800px.
{{ with .Get "link" }}<a href="{{ . }}">{{ end }} {{ with .Get "link" }}<a href="{{ . }}">{{ end }}
<picture> <picture>
<source type="image/webp" <source type="image/webp"
sizes="(max-width: 600px) 350px, 700px" sizes="(max-width: 600px) 350px, {{ $width }}"
srcset=' srcset='
{{- if ge $src.Width "350" }} {{- if ge $src.Width "350" }}
{{ with $w350.RelPermalink }}{{.}} 350w{{ end }} {{ with $w350.RelPermalink }}{{.}} 350w{{ end }}