motiejus/turbonss
1
Fork 0
Code Releases Activity

more primitive types, start with File

Browse Source
This commit is contained in:
Motiejus Jakštys 2022-02-12 12:30:50 +02:00 committed by Motiejus Jakštys
parent fe1ae55940
commit 7c63c7ab3b
1 changed files with 41 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
README.md
View File

@ -51,11 +51,10 @@ The following operations need to be fast, in order of importance:
2. lookup uid -> user. 2. lookup uid -> user.
3. lookup groupname -> group. 3. lookup groupname -> group.
4. lookup username -> user. 4. lookup username -> user.
5. lookup uid -> list of gids. 5. (optional) iterate users using a defined order (`getent passwd`).
6. (optional) iterate users using a defined order (`getent passwd`). 6. (optional) iterate groups using a defined order (`getent group`).
7. (optional) iterate groups using a defined order (`getent group`).
Indexes Indices
------- -------
Preliminary results of playing with [cmph][cmph]: Preliminary results of playing with [cmph][cmph]:
@ -66,7 +65,7 @@ BDZ: tried b=3, b=7 (default), and b=10.
* Latency for 1M keys: (170ms, 180ms, 230ms). * Latency for 1M keys: (170ms, 180ms, 230ms).
* Packed vs non-packed latency differences are not meaningful. * Packed vs non-packed latency differences are not meaningful.
CHM retains order, however, 0M keys weigh 8MB. 10k keys are ~20x larger with CHM retains order, however, 1M keys weigh 8MB. 10k keys are ~20x larger with
CHM than with BDZ, eliminating the benefit of preserved ordering. CHM than with BDZ, eliminating the benefit of preserved ordering.
Full file structure Full file structure
@ -77,12 +76,47 @@ User, Group records and their indices. All indices are number of bytes,
relative to the beginning of the file. relative to the beginning of the file.
``` ```
// /home/motiejusMotiejus.Jakstys is 16+30=46b for "my" record. const File = struct {
magic: [4]u8,
version: u3,
shells_oob: u1,
padding: u4,
num_users: u32,
num_groups: u32,
<... TBD ...>
}
```
`magic` must be 0xf09fa4b7, and `version` must be `0`. The remaining fields are
indices to further sections of the file with their sizes in bytes. All numbers
are little-endian.
What's remaining, variable-length:
1. A lookup list of shells (if `shells_oob` is True).
2. 4 indices mentioned above.
3. <...>
Primitive types:
```
const Group = struct {
gid: u32,
// index to a separate structure with a list of members
members_offset: u29,
padding: u3,
groupname_len: u8,
// a variable-sized array that will be stored immediately after this
// struct.
stringdata []u8;
}
const User = struct { const User = struct {
uid: u32, uid: u32,
gid: u32, gid: u32,
// pointer to a separate structure that contains a list of gids
additional_gids_offset: u29, additional_gids_offset: u29,
shell_here: u1, // whether it's stored "here" or in another place. Docs TBD padding: u1,
shell_len: u6, shell_len: u6,
home_len: u6, home_len: u6,
username_len: u6, username_len: u6,
@ -93,10 +127,5 @@ const User = struct {
} }
``` ```
`magic` must be 0xf09fa4b7, and `version` must be `0x00`. The remaining fields
are indexes to further sections of the file with their sizes in bytes. All
numbers are little-endian. Each field may be aligned to 64B (L1D cache size) or
4KB (standard page size), to be decided.
[git-subtrac]: https://github.com/apenwarr/git-subtrac/ [git-subtrac]: https://github.com/apenwarr/git-subtrac/
[cmph]: http://cmph.sourceforge.net/ [cmph]: http://cmph.sourceforge.net/