half-size img
parent
d4838bb558
commit
d311e0f66d
|
|
@ -76,11 +76,15 @@ figure {
|
|||
margin: 40px 0;
|
||||
}
|
||||
|
||||
figure.right {
|
||||
float: right;
|
||||
}
|
||||
|
||||
img,figcaption {
|
||||
margin: 10px auto;
|
||||
padding: 0 10px;
|
||||
height: auto;
|
||||
display: block
|
||||
display: block;
|
||||
}
|
||||
|
||||
@media (max-width:600px) {
|
||||
|
|
|
|||
|
|
@ -11,62 +11,55 @@ draft: true
|
|||
|
||||
<!-- o_ -->
|
||||
|
||||
TLDR: Modern programming languages make it very easy to add many dependencies.
|
||||
That is nice for development, but a nightmare for maintenance. Unfortunately,
|
||||
zig is following suit. I wish we could accept that adding dependencies does not
|
||||
have to be trivial. If we accept that, thanks to ubiquity of git, we may have
|
||||
almost solved the dependency problem.
|
||||
TLDR
|
||||
----
|
||||
|
||||
Modern programming languages make it very easy to add many dependencies. That
|
||||
is nice for development, but a nightmare for long-term maintenance.
|
||||
Unfortunately, zig is following suit. I wish we could accept that adding
|
||||
dependencies does not have to be trivial. If we accept that, thanks to ubiquity
|
||||
of git, we may have almost solved the dependency problem: not only for zig, but
|
||||
for everyone.
|
||||
|
||||
Adding dependencies
|
||||
-------------------
|
||||
|
||||
All of the programming languages I've used professionally, the names of which do not
|
||||
start with "c"[^1], have package managers[^2], which make "dependency
|
||||
All of the programming languages I've used professionally, the names of which
|
||||
do not start with "c"[^1], have package managers[^2], which make "dependency
|
||||
management" easy. These package managers will, as part of the project's build
|
||||
process, download and build the dependencies, which makes adding and using
|
||||
third-party dependencies easy.
|
||||
process, download and build the dependencies. So there is virtually no
|
||||
resistance to add dependencies when we need them.
|
||||
|
||||
Because C/C++ still does not have a universal package manager, not adding
|
||||
Because C/C++ still does not have a "universal" package manager, not adding
|
||||
external dependencies to C/C++ is the path of least resistance; instead, one
|
||||
relies on libraries already installed in the system. Therefore, there is a
|
||||
plethora of dependency managers that will discover but not install
|
||||
dependencies: autotools, cmake, pkg-config, and others. As a result, C/C++
|
||||
projects I've participated in usually had 0-5 non-system dependencies, whereas
|
||||
non-C/C++ projects -- tens, hundreds, or thousands[^3]. Having many system
|
||||
dependencies is painful for *every user* of the package (because they have to
|
||||
make sure the libraries, and their correct versions, are installed), so C/C++
|
||||
projects avoid having too many of them.
|
||||
|
||||
Not doing things that are easy to do requires discipline: brushing teeth,
|
||||
limiting candy intake, not adding dependencies all over the place. If it is
|
||||
easy to add dependencies and there is no discipline not doing so, the project
|
||||
will gain a lot of dependency "weight" with time.
|
||||
|
||||
{{<img src="_/2022/brick-house.jpg"
|
||||
alt="House made out of Duplo pieces"
|
||||
caption="Just like this brick house, \"modern\" package managers are optimized for building, not maintenance. Photo mine, house by my sons."
|
||||
hint="photo"
|
||||
>}}
|
||||
relies on libraries already installed in the system. There is a plethora of
|
||||
tools that will discover system dependencies: autotools, cmake, pkg-config, and
|
||||
others. As a result, C/C++ projects I've participated in usually had 0-5
|
||||
non-system dependencies, whereas non-C/C++ projects -- tens, hundreds, or
|
||||
thousands[^3]. Having many system dependencies is painful for *every user* of
|
||||
the package (because they have to make sure the libraries, and their correct
|
||||
versions, are installed), so C/C++ projects tend avoid having too many of them.
|
||||
|
||||
In Go and Python, a small number of dependencies is often a sign of care and
|
||||
quality. [mattn/go-sqlite3](https://github.com/mattn/go-sqlite3),
|
||||
[uber/zap](https://github.com/uber-go/zap),
|
||||
[apenwarr/redo](https://github.com/apenwarr/redo) and
|
||||
[django](https://djangoproject.com) are good examples. I've built and used
|
||||
these projects in a number of environments without issues. Conversely, projects
|
||||
with many dependencies often fail to build even in the environment they are
|
||||
developed and at and thus had received most testing (e.g. a specific
|
||||
OS+architecture, like `Ubuntu 16.04 x86_64`). It's even worse to do on a
|
||||
non-standard environment, no matter how trivially different (e.g. they would
|
||||
build on Ubuntu 16.04, but fail on Ubuntu 18.04), not to mention a different
|
||||
OS. This, obviously, leads to both user frustation, packagers' frustation, and
|
||||
developer long-term frustration and costs.
|
||||
these projects in a number of environments. Conversely, projects with many
|
||||
dependencies, even when pinned, often fail to build even in the environment
|
||||
they are developed at and thus had received most testing (e.g. a specific
|
||||
OS+architecture, like `Ubuntu 16.04 x86_64`). It's even worse if the
|
||||
environment, no matter how trivially, is different from the one developer is
|
||||
working at[^4]. Let's forget about a different OS or a different build system.
|
||||
Inability to build software, unsurprisingly, leads to user frustration,
|
||||
packagers' frustration, and the developers asking themselves why have they
|
||||
chosen a career in software instead of, say, farming.
|
||||
|
||||
The costs of just having dependencies are huge. I haven't done a survey and
|
||||
have only my experience to base this on (read: "many anecdotes of me failing to
|
||||
build stuff I wrote a decade ago"). But it is bad enough that I have a
|
||||
dependency checklist and am prepared to do the grunt work to save my future
|
||||
self. Here is it:
|
||||
To recap, the costs of just having dependencies are huge. I haven't done a
|
||||
survey and have only my experience to base this on (read: "many anecdotes of me
|
||||
failing to build stuff I or others wrote a decade ago"). But it is bad enough
|
||||
that I have a dependency checklist and am prepared to do the grunt work to save
|
||||
my future self. Here is it:
|
||||
|
||||
1. Does the dependency do what I want, does it work at all?
|
||||
2. Is it well written? API surface, documentation, tests, error handling, error
|
||||
|
|
@ -76,70 +69,110 @@ self. Here is it:
|
|||
4. It's system dependencies.
|
||||
5. It's transitive dependencies.
|
||||
|
||||
Assuming a "programming-language-specific package manager that does what it's
|
||||
advertised to do", the path of least resistance, when it comes to this
|
||||
checklist, is doing (1), and perhaps (2). Why bother with transitive
|
||||
dependencies or it's build complexity, if the package manager will take care of
|
||||
it all anyway?
|
||||
When working with a "programming-language-specific package manager that does
|
||||
what it's advertised to do", the path of least resistance, when it comes to
|
||||
this checklist, is doing (1), and perhaps (2). Why bother with transitive
|
||||
dependencies or it's build complexity, if the package manager takes care of it
|
||||
all anyway?
|
||||
|
||||
Except it will only when you are adding it. Package manager will not help you
|
||||
when the dependency disappears, its API changes, it stops doing what it has
|
||||
advertised and many other [problems][crash-of-leftpad].
|
||||
Except package manager will only help during the initial development, when the
|
||||
developer happily adds the package. It will work for a couple of days. Package
|
||||
manager will not help when the dependency disappears, its API changes, it stops
|
||||
doing what it has advertised and many other [problems][crash-of-leftpad]. When
|
||||
something breaks (and it inevitably will, unless it's SQLite), the work is on
|
||||
the maintainer to fix it.
|
||||
|
||||
I am trying to do all 5. If a dependency is well written, but has more
|
||||
I am following my checklist. If a dependency is well written, but has more
|
||||
transitive dependencies than I need and there is no good alternative, I will
|
||||
fork and trim it. My recent example is
|
||||
[sql-migrate](https://github.com/motiejus/sql-migrate).
|
||||
|
||||
Not doing things that are easy to do requires discipline: brushing teeth,
|
||||
limiting candy intake, not adding dependencies all over the place. If adding
|
||||
dependencies is easy (and there is no established discipline of limiting them),
|
||||
the project will tend to gain them; lots of them.
|
||||
|
||||
{{<img src="_/2022/brick-house.jpg"
|
||||
alt="House made out of Duplo pieces"
|
||||
caption="Just like this brick house, \"modern\" package managers are optimized for building, not maintenance. Photo mine, house by my sons."
|
||||
hint="photo"
|
||||
>}}
|
||||
|
||||
To sum up, the "modern" languages optimize for initial development experience,
|
||||
not maintenance. And as [Corbet says][linux-rust], "We can't understand why
|
||||
Kids These Days just don't want to live that way". Kids want to build, John,
|
||||
not maintain. A 4-letter Danish corporation made a fortune by selling toys that
|
||||
do not need to be maintained: they are designed to be disassembled and built
|
||||
anew. We are still kids. Growing up requires discipline, which is very hard,
|
||||
when candy is cheap and package managers (and disks and network, which make all
|
||||
of it possible) are as good as they are today.
|
||||
anew. We are still kids. Growing up and sticking to our own rules requires
|
||||
discipline.
|
||||
|
||||
If I may combine Corbet's views with mine: if we understand and audit our
|
||||
dependencies (all of them, including transitive ones), we will have less
|
||||
dependencies and a more maintainable system. Win-win.
|
||||
dependencies and a more maintainable system. Win.
|
||||
|
||||
Which brings us to...
|
||||
Which brings us to git submodules and git-subtract.
|
||||
|
||||
git-subtrac
|
||||
-----------
|
||||
git submodules and git-subtrac
|
||||
------------------------------
|
||||
|
||||
[`git-subtrac`][git-subtrac] manages our git dependencies (in our git
|
||||
repository) just like "classic" git submodules, but all refs of the
|
||||
A quick primer on [git submodules][git-submodule], a prerequisite to understand
|
||||
`git-subtrac`:
|
||||
* A submodule is a pointer to a particular ref in a separate repository,
|
||||
optionally checked out in our tree. For example, `deps/cmph` would contain
|
||||
all the files from [cmph][cmph]. This means that once the repository is fully
|
||||
set up (technically, the submodule is synced/updated), the build system
|
||||
(Makefiles, build.zig or what have you) can use it just like a regular
|
||||
directory.
|
||||
* The pointer to the submodule in your repository is just a tuple: `(git URL,
|
||||
sha1)`.
|
||||
* When cloning a repository that has submodules, git will not clone the
|
||||
submodules, it will just leave empty directories. We must pass `--recursive`
|
||||
for git to clone everything. Which makes sense when submodules are external
|
||||
and may not download at all.
|
||||
|
||||
Submodules were designed for adding external dependencies to a repository.
|
||||
However, using them incorrectly is way too easy, and is not fun when happens. I
|
||||
see at least these significant usability problems:
|
||||
- It is too easy to commit unintended changes to submodule, causing misery to
|
||||
others.
|
||||
- By default submodule contents (i.e. code of your dependency) lives *outside
|
||||
the repository*. This means that, with time, if dependency disappears, we
|
||||
will not be able to compile our code. Gone.
|
||||
|
||||
Because of the many usability problems of submodules, very few people use it.
|
||||
So [Avery Pennarun][apenwarr] (creator of [git-subtree][git-subtree], by the
|
||||
way) created [`git-subtrac`][git-subtrac]. `git-subtrac` bundles our git
|
||||
dependencies just like "classic" git submodules, but all refs of the
|
||||
dependencies stay in the same repository. Wait, stop here. Repeat after me: _it
|
||||
is git submodules, but all refs stay in the same repository_. I also call it
|
||||
"good vendoring". Since all the deps are in our repo, no external force can
|
||||
make our dependency unavailable, change without notice. And it will keep the
|
||||
size of the repository in check, because it's all there when you pull it.
|
||||
"good vendoring". Since all the dependencies are in our repo, no external force
|
||||
can make our dependency unavailable. And it will keep the size of the
|
||||
repository in check, because it's all there when we pull it. [`git-subtrac`
|
||||
fixes a few other submodule usability problems][apenwarr-subtrac] along the
|
||||
way.
|
||||
|
||||
Because `git-subtrac` is a vendoring tool, not a package manager, it only
|
||||
vendors but does not help building packages. Therefore, with `git-subtrac` it
|
||||
is harder to add and "make work" (build, test, add transitive deps) a
|
||||
dependency than with a language-specific package manager. Oh, what about the
|
||||
transitive dependencies?
|
||||
is harder to add and "make work" (build, test, add transitive dependencies) a
|
||||
dependency than with a language-specific package manager.
|
||||
|
||||
[`git-subtrac`][git-subtrac] does not deal with transitive dependencies. At
|
||||
least not directly. Or I am not aware of it. Ok, I haven't tried.
|
||||
|
||||
If we audit and thus understand our dependencies, we will be able to add the
|
||||
transitive ones. So perhaps git-subtrac shouldn't care?
|
||||
`git-subtrac`, just like git and submodules, does not understand "semantic
|
||||
versions". So we can't ask for "latest foo of version 1.2.X"; the developer
|
||||
will need to figure out, and hardcode, *exactly* which versions to use. Also,
|
||||
updating dependencies is not as easy as, say, in Gospeak, `go get -u ./...`;
|
||||
git will need a bit more hand holding.
|
||||
|
||||
What about Zig?
|
||||
---------------
|
||||
|
||||
Zig will have a package manager ([ziglang/zig#943][943]). I am not not very
|
||||
enthusiastic about it; can we all use git-subtrac and be done with it?. A few
|
||||
enthusiastic about it; can we all use git-subtrac and be done with it? A few
|
||||
weeks ago in a park in Milan my conversation with [Andrew
|
||||
Kelley](https://andrewkelley.me/) was something like:
|
||||
|
||||
- me: "git-subtrac yadda yadda yadda submodules but better yadda yadda yadda".
|
||||
- Andrew: "If I clone a repository that uses it with no extra parameters, will
|
||||
it work as expected?"
|
||||
- Andrew: "If I clone a repository that uses subtrac with no extra parameters,
|
||||
will it work as expected?"
|
||||
- me: "No, you have to pass `--recursive`, so git will checkout submodules...
|
||||
even if they are already fetched."
|
||||
- Andrew: "Then it's a piece-of-shit-approach."
|
||||
|
|
@ -148,15 +181,19 @@ Uh, I agree. People have not grown muscle memory to clone repositories with
|
|||
`--recursive` flag and never will, so it's impossible to adopt git-subtrac
|
||||
beyond well-controlled silos. Which is why we will have a
|
||||
yet-another-programming-language-specific-package-manager. Or at least my
|
||||
argument for using and advertising `git-subtrac` (and saving a lot of time for
|
||||
Zig folks, and a lot of inevitable misery for its users) stops right there.
|
||||
argument offering `git-subtrac` as Zig's package manager (thus saving a lot of
|
||||
time for Zig folks, and a lot of inevitable misery for its users) stops right
|
||||
there.
|
||||
|
||||
Zig has a rich standard library, therefore it does not need many dependencies
|
||||
by design. Does it *really* need a package manager?
|
||||
|
||||
Conclusion
|
||||
----------
|
||||
|
||||
Can git check out submodules when they are in the same repository, so our
|
||||
conversation of reconsidering (or not having) a Zig package manager doesn't
|
||||
stop after 5 seconds?
|
||||
When all contents of the submodules are in our repository, can git check out
|
||||
submodules too? That way, my and Andrew's conversation of reconsidering (or not
|
||||
having) a Zig package manager will have a chance to not stop after 5 seconds.
|
||||
|
||||
[^1]: Alphabetically: Erlang, Go, Java, Javascript, PHP, Perl, Python.
|
||||
[^2]: Usually written in the same language. Zoo of package managers (sometimes
|
||||
|
|
@ -164,8 +201,14 @@ stop after 5 seconds?
|
|||
in an on itself worth another blog post.
|
||||
[^3]: `go.sum` of a project I am currently involved in clocks around 6k lines.
|
||||
This is quite a lot for Go, but still peanuts to Node.js.
|
||||
[^4]: For example, they would work on Ubuntu 16.04, but fail on Ubuntu 18.04.
|
||||
|
||||
[git-subtrac]: https://github.com/apenwarr/git-subtrac/
|
||||
[linux-rust]: https://lwn.net/SubscriberLink/889924/a733d6630e3b5115/
|
||||
[crash-of-leftpad]: https://drewdevault.com/2021/11/16/Cash-for-leftpad.html
|
||||
[943]: https://github.com/ziglang/zig/issues/943
|
||||
[git-submodule]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
|
||||
[cmph]: http://cmph.sourceforge.net/
|
||||
[git-subtree]: https://git.kernel.org/pub/scm/git/git.git/plain/contrib/subtree/git-subtree.txt
|
||||
[apenwarr]: https://apenwarr.ca
|
||||
[apenwarr-subtrac]: https://apenwarr.ca/log/20191109
|
||||
|
|
|
|||
|
|
@ -15,6 +15,8 @@ care about is 700*4=2800px.
|
|||
{{ $j1400 := $src.Resize "1400x" }}
|
||||
{{ $j2800 := $src.Resize "2800x" }}
|
||||
|
||||
{{ $width := cond (eq (.Get "half") "true") "350px" "700px" }}
|
||||
|
||||
{{ with .Get "hint" }}
|
||||
{{ else }}
|
||||
{{ errorf "missing value for param 'hint': %s" .Position }}
|
||||
|
|
@ -29,7 +31,7 @@ care about is 700*4=2800px.
|
|||
{{ with .Get "link" }}<a href="{{ . }}">{{ end }}
|
||||
<picture>
|
||||
<source type="image/webp"
|
||||
sizes="(max-width: 600px) 350px, 700px"
|
||||
sizes="(max-width: 600px) 350px, {{ $width }}"
|
||||
srcset='
|
||||
{{- if ge $src.Width "350" }}
|
||||
{{ with $w350.RelPermalink }}{{.}} 350w{{ end }}
|
||||
|
|
|
|||
Loading…
Reference in New Issue