replace member strlen with getgr_max and getpw_max
This way we can use buffers in both Zig and C, without the performance or size penalty.
This commit is contained in:
32
README.md
32
README.md
@@ -111,7 +111,7 @@ A known implementation runs id(1) at ~250 rps sequentially on ~20k users and
|
||||
|
||||
To better reason about the trade-offs, it is useful to understand how `id(1)`
|
||||
is implemented, in rough terms:
|
||||
- lookup user by name ([`getpwent_r(3)`][getpwent_r]).
|
||||
- lookup user by name ([`getpwent_r(3)`][getpwent]).
|
||||
- get all gids for the user ([`getgrouplist(3)`][getgrouplist]). Note: it is
|
||||
actually using `initgroups_dyn`, accepts a uid, and is very poorly
|
||||
documented.
|
||||
@@ -154,6 +154,9 @@ OFFSET TYPE NAME DESCRIPTION
|
||||
40 u64 nblocks_users
|
||||
48 u64 nblocks_groupmembers
|
||||
56 u64 nblocks_additional_gids
|
||||
64 u64 getgr_max
|
||||
72 u64 getpw_max
|
||||
80 [48]u8 padding
|
||||
```
|
||||
|
||||
`magic` is 0xf09fa4b7, and `version` must be `0`. All integers are
|
||||
@@ -165,6 +168,10 @@ number of bytes. However, interpreting `[2]u6` with `xxd(1)` is harder than
|
||||
interpreting `[2]u8`. Therefore we are using the space we have to make these
|
||||
integers byte-wide.
|
||||
|
||||
`getgr_max` and `getpw_max` is a hint for the caller of `getgr*` and
|
||||
`getpw*`-family calls. This is the recommended size of the buffer, so the
|
||||
caller does not receive `ENOMEM`.
|
||||
|
||||
Primitive types
|
||||
---------------
|
||||
|
||||
@@ -279,20 +286,17 @@ respectively. Membership IDs are packed — not necessitating random access, thu
|
||||
suitable for compression.
|
||||
|
||||
- `groupmembers` consists of a number X followed by a list of offsets to User
|
||||
records, because `getgr*_r` returns pointers to membernames, thus a name has
|
||||
to be immediately resolvable.
|
||||
records, because `getgr*` returns pointers to membernames, thus a name has to
|
||||
be immediately resolvable.
|
||||
- `additional_gids` is a list of gids, because `initgroups_dyn` (and friends)
|
||||
returns an array of gids.
|
||||
|
||||
Each entry of `groupmembers` and `additional_gids` starts with a varint N,
|
||||
which is the number of upcoming elements. Then depending on the type:
|
||||
which is the number of upcoming elements. Then N delta-compressed varints,
|
||||
which are:
|
||||
|
||||
- **additional_gids** stores N delta-compressed varints. These varints
|
||||
correspond to a list of gids.
|
||||
- **groupmembers** stores a total length of the member names, followed by N,
|
||||
followed by N offsets. These are byte-offsets to the User records in the
|
||||
`users` section. Having the length of membernames saves some CPU cycles when
|
||||
decompressing the members in the hot path.
|
||||
- **additional_gids** a list of gids.
|
||||
- **groupmembers** byte-offsets to the User records in the `users` section.
|
||||
|
||||
Indices
|
||||
-------
|
||||
@@ -339,7 +343,7 @@ Each section is padded to 64 bytes.
|
||||
|
||||
```
|
||||
SECTION SIZE DESCRIPTION
|
||||
header 64 see "Turbonss header" section
|
||||
header 128 see "Turbonss header" section
|
||||
bdz_gid ? bdz(gid)
|
||||
bdz_groupname ? bdz(groupname)
|
||||
bdz_uid ? bdz(uid)
|
||||
@@ -365,7 +369,7 @@ Section creation order:
|
||||
1. ✅ `groupmembers` requires `users`.
|
||||
1. ✅ `groups` requires `groupmembers`.
|
||||
1. ✅ `idx_*`. requires offsets to `groups` and `users`.
|
||||
1. Header.
|
||||
1. ✅ Header.
|
||||
|
||||
[git-subtrac]: https://apenwarr.ca/log/20191109
|
||||
[cmph]: http://cmph.sourceforge.net/
|
||||
@@ -374,6 +378,6 @@ Section creation order:
|
||||
[data-oriented-design]: https://media.handmade-seattle.com/practical-data-oriented-design/
|
||||
[getpwnam_r]: https://linux.die.net/man/3/getpwnam_r
|
||||
[varint]: https://developers.google.com/protocol-buffers/docs/encoding#varints
|
||||
[getpwent_r]: https://www.man7.org/linux/man-pages/man3/getpwent_r.3.html
|
||||
[getpwent]: https://www.man7.org/linux/man-pages/man3/getpwent_r.3.html
|
||||
[getgrouplist]: https://www.man7.org/linux/man-pages/man3/getgrouplist.3.html
|
||||
[getgrid_r]: https://www.man7.org/linux/man-pages/man3/getgrid_r.3.html
|
||||
[getgrid]: https://www.man7.org/linux/man-pages/man3/getgrid_r.3.html
|
||||
|
||||
Reference in New Issue
Block a user