[Toybox] A note on my "todo list".

Thu Nov 30 14:41:58 PST 2023

I say "it's on the todo list" a lot, which I'm guilty about because it's not
public, but my todo list isn't _secret_. It's just unintelligible to most people
and spread across a _bunch_ of different places.

Attached is the main top-level todo.txt from my ~/toybox/toybox work directory
that I'm copying todo items into as I close tabs in which was editing or testing
something, or had a man page line highlighted to remind me of something, or a
relevant historical git log entry pulled up that has follow-up work... (Hence
why rebooting my laptop improperly is annoying every time it happens.)

This file does not include the todo2.txt, todo.today, and todo.now files
adjacent to it in that directory, nor the... 137 *.patch files in the same
directory (several of which I've already applied and need to cull, there's
debris everywhere this is just some of the known non-debris), the various files
like "dd.txt" and "todo-microcom.txt" with command-specific notes, the 95 files
in the "todo" subdirectory, the various TODO notes in the source, the multiple
changes in the tree (git diff | diffstat says 67 files changed, 674
insertions(+), 345 deletions(-)) and that's not counting the untracked files (11
untracked command.c files just in toys/pending, with various partial
implementations of zcip, tput, fbsplash, mkisofs, ar, at...). And back at the
top level the various test.c route.c tsort.c files that take different
approaches than the merged command, and stuff like arbys.c which is the
red-black tree code I wrote in... 2018 apparently, but have not yet installed
into lib and made commands use... And the www directory has a BUNCH of files in
it for FAQ entries I should write, or a rewrite of license.txt and various other
0BSD promotion/explanation work I should do...

And that's just one (very dirty) git repo: cd up one directory into merely
"~/toybox" and there's another todo.txt and todo directory, and notes.txt, and
"youtube" and "video" subdirectories in which I've written a bunch more files
(attaching the "mmu.txt" file out of there which I need to finish and record for
example), and a "pending" directory with stuff I hope to add in future... And of
course there are several more toybox checkouts with different projects
half-finished in each one adjacent to my "main" tree:

$ ls */main.c
clean2/main.c clean7/main.c github/main.c kl6/main.c toybox/main.c
clean3/main.c clean8/main.c kl2/main.c kl7/main.c
clean4/main.c clean/main.c kl3/main.c kl8/main.c
clean5/main.c clean.old/main.c kl4/main.c kl9/main.c
clean6/main.c debian/main.c kl5/main.c kleen/main.c

Going through and processing the unfinished work in those directories is one of
those things adjacent to tab closing: the directories proliferate and need
squashing. Alas, opening todo tabs faster than I finish them is how I get buried
in tabsplosion in the first place, it manifests in many ways...

All that was just the ~/toybox directory, of course. Outside it there ~/todo.txt
(92 lines) and ~/notes.txt, ~/linux/linux has its own multiple ~/linux/clean
and ~/linux/patches and... Plus ~/mcm (musl-cross-make) and ~/talks and...

I need to reread my old blog entries to seen what I mentioned there that's
gotten dropped (I mention stuff there so I can go back and find it again). I
should reread the mailing list archive for similar reasons. And alas github has
issues and pull requests. And various distros ship toybox with patches they've
never sent me...

There's a reason "collate todo lists" is on multiple todo lists. Lotta shoveling
goes on behind the scenes that doesn't immediately result in checkins...

Rob
-------------- next part --------------
static vdso?

Toybox tries to be NOMMU friendly, which means it runs on embedded systems
without an active "memory management unit".

MMU provides virtual addresses via page tables and a TLB

On nommu, there are only physical memory addresses.
 - every program sees the same address, so if one process is using 0x12345
 another process can't.
 - everything must be relocatable code, -fpic like shared libraries.
 - no swap
 - no deferred allocation
 - no copy on write
 - no page remapping
 - memory fragmentation is global, not just within a process
 - can't fork(), must vfork()

- FDPIC vs PIE
 - readelf -a (-d)
 - cat /proc/self/maps (smaps, smaps_rollup)
 - mmap()
 - file mappings
 - anonymous mappings
 - system mappings (stack+environment, heap, vdso)

 - Each ELF object has:
 - which segment it belongs to
 - start address (offset within segment)
 - length (number of bytes)

 - Four main ELF segments:
 - text, rodata, data, bss
 - text: executable segment (all your functions)
 - rodata: read only data (string constants, vars declared "const").
 - data: writeable global variables initialized to a value
 - bss: writeable global variables initialized to zero
 - anonymnous mapping, not file mapping.
 Don't waste space in file storing zeroes. Just length.

 - dynamic linking crap
 - plt, got, 

 - C++ crap
 - init/fini - constructors and destructors. Array of function pointers.
 - eh_frame - stack unwinding

 - stack and heap dynamically allocated
 - heap can be mmap() but traditionally brk()
 - mmu can collate as long as virtual address space available
 - on 64 bit systems, can leave ENORMOUS gaps
 - environment usually at end of stack (argv, envp)

 - shared libraries
 - each has own text, rodata, data, and bss segments!
 - plus PLT (Procedure Linkage Table) and GOT (Global Offset Table)
 - and "interp"
 - 

 - in conventional ELF, linked at compile time relative to 0, you know what
 address everything is loaded at. Each process has its own virtual address
 space, so those address are always available.
 - in PIE, there's a "base pointer".
-------------- next part --------------
Post updated kernel patches to linux-kernel yet again

Poke coreutils about cut -DF

tar --hole-detection={seek,raw} # implies --sparse
 - redo test for FS portability

genericize copy_tempfile?
 kl8/ #06-11-2023

echo should produce atomic output

Replace freebsd-13.raw with freebsd-14.raw, delete old to save space

factor out yes/logpath string collation (or writev iovec?)

xmodem sx/rx actually ymodem

wc -L

microcom probe for serial
 setserial is using ioctl(TIOCGSERIAL);

hash table stuff: hsearch() out of libc ?
 - musl src/search/hsearch.c

Finish microcom cleanup: todo-microcom.patch and elliott's menu

Standardize help text (sysctl rfkill etc)
 http://landley.net/notes-2023.html#06-11-2023
 should [-x BLAH] become "-x BLAH text text" or "-x text BLAH text" ?

no acceptable grep autoconf test in make build

fix count.c restrict nonsense warning

fix initramfs shell
 https://github.com/landley/toybox/issues/459

orange pi
 https://uthings.uniud.it/building-mainline-u-boot-and-linux-kernel-for-orange-pi-boards
 https://github.com/orangepi-xunlong/linux-orangepi
 https://github.com/orangepi-xunlong/u-boot-orangepi

make tr utf8
 but not collation because https://www.unicode.org/reports/tr10/tr10-49.html
 http://databasearchitects.blogspot.com/2016/08/equivalence-of-unicode-strings-is.html

./toybox sed --help
Single line output. Wrong.

sed -s with toybox treats 'q' as skipping to the next file: gnu/sed exits.

redo tsort with Ray Gardner's (Knuth's) algorithm.

https://man7.org/linux/man-pages/man5/keymaps.5.html
 loadkeys loadkmap dumpkmap scankey
 http://kbd-project.org/docs/scancodes/scancodes-1.html
 https://github.com/legionus/kbd/blob/master/data/keymaps/i386/qwerty/defkeymap.map

Upgrade test suite so gentoo can run it.
 Request filesystem type, umount -l.
 ldd chroot https://github.com/landley/toybox/commit/e70126eabef8
Finish lspci -x fallout.
 Check compression? https://github.com/landley/toybox/issues/386
 http://lists.landley.net/pipermail/toybox-landley.net/2023-April/029520.html
Finish cgroup stat support.
 https://github.com/landley/toybox/issues/423
Yifan Hong's continuing tar weirdness:
 https://android-review.googlesource.com/c/2536710
Peter Maydell qemu Malta patch?
Tom Lisjac (and previous guy) want scripts/root/dynamic
 https://github.com/landley/toybox/issues/418
David Legault, fold tests. (Promote fold?)
 https://github.com/landley/toybox/issues/424
vmstat for zhmars
 https://github.com/landley/toybox/issues/422
sizeof(toybuf)
 https://en.cppreference.com/w/c/language/_Alignas
fix sh2eb mkroot build (toolchain and kernel)
gzip --rsyncable
 implement deflate, implement rsync...
Ongoing cleanup of mdev.c started on plane due to /sys/block poke.
 http://lists.landley.net/pipermail/toybox-landley.net/2023-April/029525.html
Finish the cp -s work so I can do install -T
Try to beat a multi-console thing out of mkroot+qemu to test oneit change
 http://lists.landley.net/pipermail/toybox-landley.net/2023-April/029531.html

linebuf/iovec() shenanigans
 http://lists.landley.net/pipermail/toybox-landley.net/2023-October/029843.html
 TOYFLAG_LINEBUF, dprintf(1, blah) = actual_xprintf()
 grep LINEBUF toys/*/*.c
 Debian:
 $ while true; do echo hello; sleep .1; done | base32 | toybox time head -c 1
 real	523.443
 user	0.1534
 sys	0.000

Line editing
 - backspace is funky
 $ echo -e '\t\bx'
 x
 $ echo -e '\t\b\b\bx'
 x
 $ echo -e 'r\xcc\x88'
 r̈

busybox faq
https://git.busybox.net/busybox/tree/docs/busybox.net/FAQ.html?h=1_3_stable#n374

tests/files/utf8/wide.txt

toybox $PATH indicator: before others, after others, don't recurse at all?
 $ rm -f toybox && cp ../toybox . && sudo chroot . /toybox sh
 # ls
 sh: ls: No such file or directory

More relative path stuff:
 cp -s, ln -s

sort by llast column with key = -1
 https://lists.gnu.org/archive/html/bug-coreutils/2006-12/msg00008.html

ls -k
http://lists.landley.net/pipermail/toybox-landley.net/2023-August/029681.html

Test passwd, chsh, groupadd, groupdel, useradd, userdel
Move hashes to lib and have lib/passwd.c do internal hashing
 - Freebsd has: des blf md5 sha256 sha512
 - md5sum, sha256sum, sha512sum
 - des?
 - blowfish?
 - crypt_set_format does what?
New crypt() implementation in lib/hash.c
 - __attribute__((__weak__))

expr: finish and promote
 - expr 1234567 : 1234
 4
 - and sub-match?
 - when is result string vs number?
 - Just always keep result as string and itoll() again?
 - expr 3m + 2k ?
pgrep "space name"
 $ ln -s $(which sleep) '(space name)'
 $ ./$space\ name$ 100 &
 $ cat /proc/20048/stat
 20048 ((space name))
rm -r filehandle exhaustion (and test!)
 - only zap if ".." stats the same
 - if ->parent && stat(newfd, "..") same_inode as saved, close parent fd.
 - no need to special case symlinks, etc.
 - maybe save first ~25 for performance anyway? (How to count depth?)
 - no traversal-wide data
 - if drill back down from top fails... then what?
 - drill back down from last saved fd traversing up: always one at top
 - warn and return
 - are we recursing stack when recursing nodes? (Shouldn't be...)
 - awkward 2-call thing is to avoid stack recursion. Document that.
 - document rules: when is parentfd alive? (this during comeagain?)
cp filehandle exhaustion (source _and_ dest).
 - cd .. is same directory?
 - extra is already destfd, but dest dev/ino = 128 bits
 - any way to allocate nodes with extra space at start so no realloc?
 - toys.dirtree_extra? (But... only needed for directories? Hmmm...)

 - when is realloc safe?
does failed regcomp() need regfree()?
in toys.h struct toy_context evaluate which fields should move up into zeroing

hexdump -C and hd
enh:
my "dream" hexdump would be hexdump(1) with -C as the default (so i didn't need the hd symlink) and xxd's -r and od's -t functionality [and i guess hexdump(1)'s -e for the people who use that].
my notes say we needed -n#, -s#, -C, -e, -f FILE, and -v for hexdump ... but code search shows no users in AOSP. i can see some internal uses, with -C -n 128, -s 128 -n 8 -e 'blah %d', and -v -e '1/4 "%08x"' -e '"\n"' being representative examples of the kinds of things being done. ("representative" in the sense of "if you can do those, you can do everything".)

github issues
github pulls
toybox/*.patch

vmstat: still outstanding issues (github?)

test with linux nolibc? Does anything build?

notstdio() usage outside of xwrap.c: probably all bad?

overlayfs merged bug
 http://lists.landley.net/pipermail/toybox-landley.net/2022-September/029195.html

Linux kernel module support
 https://github.com/landley/toybox/issues/351
 net dummy module?
 Can the block layer be a module? (Hmmm, mount /dev/?da wouldn't.)

tar --wildcard

rm -r infinite depth
 test: while true; do mkdir -p a b; mv a b; mv b a; done

test suite running scripts/test.sh
 scripts/root/test
 - squashfs?
 toysh:

git.c

make all TEST_HOST pass
 grep/sed in testcmd as well as txpect?

TEST_HOST filesystem portability
 tmpfs, ext4, ecryptfs, btrfs
 - gentoo test failure, what fs?

Ensure debian patch list dealt with?
 - who else packages toybox: gentoo, freebsd

modules in mkroot

ls --sort=icase
 case insensitive
 -X extension
 -f vs -U
 fallback sorts?

debootstrap
sudo scripts/record-commands debootstrap beowulf boing
$ awk '{print $1}' log.txt | sort -u | xargs
apt-config arch-test basename cat chmod chown chroot cp cut debootstrap dirname dpkg dpkg-deb gpgv grep head id ln ls mkdir mknod mv perl rm sed sha256sum sort tar touch tr umount uniq unxz wc wget
$ grep wget log.txt
$ sudo env -i USER=root TERM=linux SHELL=/bin/bash LANG=$LANG PATH=/bin:/sbin:/usr/bin:/usr/sbin unshare -Cimnpuf chroot beowulf

httpd.c:
 - mime() should be file output? (Or shared lib/ plumbing?)
 - isunder(file, dir) in lib?
 - only httpd right now
 - no cgi

host.c interactive nslookup wrapper discussion with Moritz Weber.

Move strany() from sh.c to lib: grep 'strcmp.*strcmp' toys/*/*.c

realpath --relative-to
 0001-Change-relative_path-so-from-ends-with.patch

diff
 Triage man page for flags
 in print_hunk:
 -b/B/I/E can't set return code until we've seen if it's whitespace only?
 Algorithm needs to be bidirectional.
 Peel lines of start/end alternating.

lib/password.c
 pending: groupdel chsh sulogin groupadd userdel useradd
 !pending: passwd su login mkpasswd

toys/*/sh.c
 - what did I leave off in the middle of?
 - 11 branches! And counting! Grrr...

mkroot build
 lfs-bootstrap.hdc control image (~/www/aboriginal/downloads)

mount --make-rprivate and friends?
 mount --make-X mountpoint: shared,slave,private,unbindable, and r* for each

httpd - hardening
 getline length limit
 timeout -i
 virtual domain?
 cgi

Collect kernel patches
 replace kernel/time/timeconst.bc


llvm uninitialized variable checker: CFLAGS="-ftrivial-auto-var-init=pattern"
 - on top of ASAN I guess?

llvm-hexagon toolchain and qemu-system hexagon
 https://www.openwall.com/lists/musl/2023/09/27/4

7/23/21 enh: quote for finishing toys used in AOSP

Auto-test in mkroot init script (run an externally provided thing).
Collate 2013 and 2019 talks with youtube channel somehow? (Simplest Linux?)
FAQ update for cross compiler binaries.
TODO: htop (top -h)
TODO: test "generated" living somewhere else.
TODO: FAQ entry on toolchains

Upload videos to youtube, announce on linux-embedded. :)
 ~/youtube/youtube/*.txt

gpiod cleanup?
 mkroot needs the wordwrap escape in both places?
 video on cleanup toys/*/fsfreeze.c
 linux/fs/ioctl.c FSFREEZE
 id -g nobody, finish tar --owner name:number so new tests.tar passes
 toybox wget
 lib/password.c
 toys/other/openvt.c (ate toys/other/chvt.c and promoted out of pending)
 toys/posix/cmp.c (so TEST_HOST passes)
 tests/rm.test added "poke the deep ones" but PAINFULLY slow?
 time ../toybox sh -c 'for i in {1..1000}; do mkdir $i && rmdir $i && echo $i || break; done'
 0001-Change-relative_path-so-from-ends-with.patch
 make sure generated/build.sh works
 toys/other/nbd_client.c was calling fork(), finish fixing
 confirm sh2eb works
 toys/*/microcom.c
 pending/BLAKE2/ref
 patreon kvm -m 4096 -smp 2 -cdrom devuan_chimaera_4.0.0_amd64_desktop-live.iso -drive format=raw,file=blah.img -hdb devuan_chimaera_4.1_0_amd64_pool1.iso
 Fix "open directory as file with no error" issue in loopfiles.
 Optstr with <1>2 should otherwise stop at first argument.
 - doesn't need to say &
 grep ', "[<>][0-9][^a-zA-Z]*"' toys/*/*.c | less
 chmod +x tests/date.test ?
 finish htop
 Is everything good with copy_file_range now?
 Splitting scripts/make.sh into scripts/genheader.sh?
 hexagon toolchain ~/mcm/clean/ccc/hexagon-linux-musl-cross/bin
 linux patch for $CROSS-cc, devtmpfs
 Elliott wants me to reverse GLOBALS() parse order

$ man bash
landley at driftwood:~/jcore/qflow$ x() { while $# -gt 0; do echo "$1"; done; }
landley at driftwood:~/jcore/qflow$ x a b c d e
bash: 5: command not found
landley at driftwood:~/jcore/qflow$ x() { while $# -gt 0; do echo "$1"; shift; done; }
landley at driftwood:~/jcore/qflow$ x() { while [ $# -gt 0 ]; do echo "$1"; shift; done; }
landley at driftwood:~/jcore/qflow$ x a b c d e
a
b
c
d
e
landley at driftwood:~/jcore/qflow$ ( X="one two three"; do env -i ${X:+X="$X"} env
bash: syntax error near unexpected token `do'
landley at driftwood:~/jcore/qflow$ ( X="one two three"; env -i ${X:+X="$X"} env )
X=one two three
landley at driftwood:~/jcore/qflow$ ( X="one two three\\"; do env -i ${X:+X="$X"} env
bash: syntax error near unexpected token `do'
landley at driftwood:~/jcore/qflow$ ( X="one two three\\"; env -i ${X:+X="$X"} env)
X=one two three\
landley at driftwood:~/jcore/qflow$ ( X="one two three\\"; env -i ${X:+X="$X"} env) 
Release TODO:
 - www/news.html
 - www/design.html
 - www/faq.html
 - www/roadmap.html
 - scripts/mkroot.h to stopping point
 - toys/other/readlink.c (tests/readlink.test, tests/realpath.test)
 - fix scripts/mkstatus.py (pending vanished?)
 - fix microcom and lib/tty.c (lib/lib.h toys.h)
 - lib/password.c cleanup
 - fix main.c help comment (toybox insmod --help != toybox --help)
 - toys/net/ftpget.c
 - toys/other/nbd_client.c (NOMMU support)
 - toys/*/swapon.c (add -a, todos at top)
 - toys/*/vmstat.c (80 chars?)
 - toys/*/sort.c (-h)



 - tests/cut.tests tests/find.test tests/fmt.test tests/ifconfig.test
 tests/paste.test tests/sort.test tests/test.test tests/timeout.test
 # - scripts/runtest.sh pending change?

Minor cleanups:
 toys/*/{umount,truncate,pmap,reset}.c
 - toys/posix/cksum.c

Punt:
 - www/license.html
 toys/*/flashcp.c (lib/xwrap.c xxlseek())
 toys/*/mount.c toys/*/umount.c
 toys/other/chroot.c
 toys/other/inotifyd.c
 toys/*/taskset.c
 toys/*/watch.c (flushing)

FAQ: Trusting trust
 - Ken's Thompson's ACM lifetime achievement award
 - mod compiler to recognize login program and insert new exploit
 - mod compiler to recognize compiling ITSELF and insert exploits.
 - Compile old exploitless source with new dual-exploit compiler binary
 - produces exploited binaries with no exploits anywhere in source.
 - ship clean source to berkeley with hacked binaries, login "kt" in BSD.
 - Countering trusting trust
 - minimal build environment (4 packages: linux, toybox, musl, tinycc)
 - must natively build to arbitary complexity (LFS -> AOSP and beyond).
 - binary audit minimal build environment (vericode, etc).
 - redo periodically
 - Note: new languages not written in C need auditing.
 - llvm->c compiler from Julia

Minimal native development environmnet
 - counters Trusting Trust
 - eliminates cross compiling: only base needs cross, rest native
 - llvm->c compiler from Julia


musl faq entries:
 taskset: sched_getaffinity() doesn't exist because Rich doesn't like it.
 - removed in a specific commit
 grep can't handle embedded NUL because Rich explicitly refused to implement.
 Rich explicitly refuses #ifdef __MUSL__ because his library needs no workarounds.

Zap toys.envc and friends, shell didn't use it. Does anybody?

man 3 setbuf (setlinebuf)
expr -> $((a=4)) precedence climbing algorithm

sudo

dpkg-query -L mtd-utils

modprobe and depmod

armv5l
 # ifconfig eth0 10.0.2.15
 ifconfig: bad argument '.0.2.15' (see "ifconfig --help")

Replace get_line() with getline()

dlist_add_nomalloc: two void * typecasts and awkward name
 - make struct dlist a struct that can be first member

struct arg_list should have void *, not char *
 - struct slist_wrap

Why does help -au show "true" and "test" near the start? (But false later?)

gzip help text

getopt vs getopts "point of getopt(1) over the getopts built-in is long options"
 getopts sets environment variables, must be builtin

du.test: symlinks inlined using zero blocks
 - filesystem-specific behavior

test_chattr, getfattr, and setfattr - ext2-isms. Genericize ioctls.
 - test fs type in test

expr has a pending patch (pull 148) to fix memory management
 - but $(( )), bash man page "arithmetic expression" section

cp -s relative paths not all working right
 cd ~/toybox
 mkdir bang
 cd bang
 cp -rs ../www/. .

https://linuxize.com/post/pstree-command-in-linux/

https://blog.lizzie.io/linux-containers-in-500-loc.html
https://blog.nelhage.com/2010/08/write-yourself-an-strace-in-70-lines-of-code/
 https://github.com/nelhage/ministrace

README.md
markdown https://github.com/landley/toybox/issues/128
 ~/markdown

> > for i in BC DIFF EXPR GROUPADD GZIP ROUTE SH TR USERADD VI WGET
>
> fwiw, that's remarkably similar to the AOSP list... bc, diff, expr,
> gzip, sh, and tr.

showkey setkeycodes
 KDSETKEYCODE
 http://lists.landley.net/pipermail/toybox-landley.net/2019-September/010900.html

lowhang: host expr dd diff tr less route wget init fdisk cols fold join tftp
unexpand hd nfsmount rdate arping arp man

echo and printf both have \c (and third one? Check patch then find list post.)

make root web page
 sudo chroot root/host/fs /init
 make help
 FAQ entry

Cross compiling FAQ entry

New scripts/mcm-buildall.sh convenience script to build gcc+musl cross compiler
toolchains for various targets (arm, x86, mips, m68k, s390, microblaze)
using <a href=https://github.com/richfelker/musl-cross-make>musl-cross-make</a>,
and a new scripts/cross.sh to easily cross compile for one or more targets
using the resulting compilers. (This works with make root</>, make
toybox, and other builds understanding the CROSS_COMPILE environment variable.)


setkey - linux/kd.h:#define KDSETKEYCODE	0x4B4D
 - man 4 console_codes
showkey - shows what's assigned to a specific pressed key

toplog - text log of commands using more than 50% of one CPU.
 cmdname %x, cmdname waiting, cmdname done
 filter: -thunderbird

May 3,2019 blog rumination on large data block getline()

ls needs two space gaps because combining characters are insane

mv toys/*/* to toys/* (collapse subdirs but leave pending)
 - menuconfig would then have toys, pending, and options

android is using from pending: bc, dd, diff, expr, lsof, and tr.
 update the readme already

xargs
 add -P (android needs it)
 0001-xargs-add-max-args-synonym-o-option-and-fix-p.patch

tar xz not autodetecting
 https://github.com/landley/toybox/pull/138

DIRTREE_STATLESS and errno:
 http://lists.landley.net/pipermail/toybox-landley.net/2019-August/010837.html

patch.c:

>> This supports patches that have timestamp formats like
>> "--- a/libgomp/config/posix/time.c	Thu Dec 24 11:24:47 2015
>> -0500"
>> which do not begin with a digit. An example is the musl support patches 
>> for GCC 5 at https://github.com/GregorR/musl-gcc-patches , which seem 
>> to have been generated by Mercurial.
$ diff -u sub sub2
diff -u "sub/one\ttwo" "sub2/one\ttwo"

--- "sub/one\ttwo"	2019-08-28 16:57:21.431920511 -0500
+++ "sub2/one\ttwo"	2019-08-28 16:57:41.379919636 -0500
@@ -1,5 +1,4 @@
 one
 two
-three
 four
 five

nbd new style protocol, and options

factoring:
  lib/base.c - the functions used by main()
  is there anything in main() that doesn't need to be?
  scripts/ the minimal make plumbing vs non-make plumbing.
    split generated vs build

dlist_add_nomalloc dlist_add
  dlist member pointer should be void
  dlist_add_nomalloc -> dlist_add
  dlist_addwrap()

cpio xattr
tar --xattrs (--xattrs-exclude= --xattrs-include=)

tftp, tftpd
httpd
route
toysh

genericize hardlinks:
  struct {} hlx hardlinks from tar
  cpio, zip

Make the mkroot tests work already
  quick and dirty:
    external make, external bash, external route
    what else? make tests

Fix test suite not to source tests/*.test
  Need to finish up git diff tests/ first

xpopen_wrap(char **cmd, int fd)
  make tar use it
  make man use it

# tar --restrict xvzf

Expecto

tests: replace SKIP_HOST with toyonly
  - blkid: more test_host

partprobe: new API BLKPG_ADD_PARTITION
Finish/test su and login
getty pending/sulogin
  PPP handoff, https://tools.ietf.org/html/rfc1661#section-3.4 (ala bsd getty)

lib/lib.c format_iso_time()
  - seconds instead?
  - gazillion users of %F %T variants that... aren't. grep -r localtime toys
    

yocto toybox patch
  yocto toybox build
  http://lists.landley.net/pipermail/toybox-landley.net/2018-November/009842.html
  using fsck.c, getty.c, test.c, tftp.c, and tftpd.c all out of pending,

CELF proposals?

rename xparsemillitime->xparsems see if there are more users?
find multiple {} and environ_bytes()? (find -exec +)

Review Hulbert DM, get reference(s), write up proposal.
  samsung, sony, who is Lipi Lee?
  JCI? (Ha!)

jci: bash 3.2.57, fdisk, resize2fs, rsyslog, screen, oprofile, parted, wget

sntp test
  server on 10.0.2.15
  TRAP reset time from 10.0.2.2 (host)

rdate.c

make test_date swiss cheese. (See also recent commit.)
  post about date testing
  date --date=1:2:3 vs date --date=98-7-6 zero or current?
toys/other/mkdosfs.c
nbd_server.c
rfc 3164: syslog (pending/syslogd.c)
  - rfc 5424 (new 3164), 5425 (tls), 5426 (udp), RELP, systemd journal,
    buffer output if receiver not ready
    https://www.rsyslog.com/doc/relp.html (and wikipedia page)

  https://en.wikipedia.org/wiki/Rsyslog
anything else xrecvwait()?
top -H not grouping threads right, thr.c
  go through top man page, any more options? (-f ?)
  htop bars, colors?
  what's eating all the CPU time? Faster?
  top units vary (human_readable problem)
  400%cpu confused makepost, 4xSMP? htop?

bc.c cleanup
ps ax (vs ps -x)
patch fuzz
route.c redo rtnetlink, android commit, github pull
echo -e '\033[?7h'
./qemu-i686.sh -nic user,hostfwd=:127.0.0.1:12345-:22
iotop swapin field
netcat only one codepath (see commit, proposed commit)
dhcp/zcip = same file, automatic fallback option

yocto upstreamery:
  http://cgit.openembedded.org/meta-openembedded/tree/meta-oe/recipes-core/toybox
    Eduardas Meile <eduardas.m at fods.com>
    require toybox.inc
    SRCREV = "123456789abcdef"

buildroot toybox patch
RFC writeup for class E and multicast

grub qemu install, partition toggle (hda partition/format loopback script?)

sntp -M server
that code review

Submit ping range patch to kernel again
  https://twitter.com/b0rk/status/1094297731546386437
 * Note: ping_group_range should never have existed. To disable it, do:
 *   echo 0 $(((1<<31)-1)) > /proc/sys/net/ipv4/ping_group_range
 * (Android does this by default in its init script.)

  Submit initramfs devtmpfs mount patch to kernel again

patch fuzz factor, git rename/copy/delete support

mkfs.vfat, genvfatfs, mtools

hexdump
 * -n# -s# -C -e -f FILE -v

# mkroot build not noticing 118 commits in --version? (git describe --tags)
#   - because git wasn't in $PATH

watch -> less -> edit

Text editor:
  nano
  joe
  vi
  emacs

nut: dhcpd, tftpd


deflate, zip

rsync
https://linuxconfig.org/examples-on-how-to-use-rsync-for-local-and-remote-data-backups-and-synchonization
https://shallowsky.com/blog/linux/cmdline/rsync-include-exclude.html



screen

htop

sntp server, receiver

android/core
  toolbox: grep, getevent, r, getprop.cpp
  cpio
  logcat
  run-as - sudo
  

BroadcastSender.c BroadcastReceier.c

collate --color isatty() logic. (always/never/auto)

For release:
  test.c promote (PR 47 100 102)
    http://lists.landley.net/pipermail/toybox-landley.net/2018-September/009666.html
  watch.c promote

commit scripts/genconfig.sh (prlimit fix)

lowhang:
  sntpd
  httpd, wget
  tftp, tftpd
  route
  strace sudo flex gzip htop mtd ntp ar nfsmount smbmount
  nbd-server tcpdump gzip zip arp arping ftpput ftpd
  tar expr

  sh-history vi microemacs joe screen

  deflate RFC printout
  arp rfc printout
  dhcp printout (rfc 3927)

# follow argv[0] symlinks until one's recognized
  - todo: toybox ./name shouldn't follow symlink, only top level? Hmmm...

cleanup environ_bytes()
make xparsetime() return ms like millitime()

status.html:
uncategorized: crc32 fmt uuidgen dhcp6 ipaddr iplink iproute iprule iptunnel toysh -sh -toysh traceroute6

Add toybox to buildroot
  - with gazillion CONFIG things (search for busybox)

fix ulimit

rm infinite descent

migrate sed and patch to loopfile_replacelines?
  loopfile_lines() with -i behavior?

nfsmount:
  lkml.iu.edu/hypermail/linux/kernel/1606.1/01115.html

netcat logger
ratelimit
git clone https://lore.kernel.org/lkml/0

find x = x "may be unused" assignments:
  grep '[^a-zA-Z0-9]\(..*\) = \1[,; ]' toys/*/*.c
  -Wmaybe-uninitialized not supported by llvm

sudo netstat -ltnp

mount | column -t
  column -t -s:
multitail?

cd - # OLDPWD not set

sudo !! # run last command as root

> what i miss more is not having a good way to check for expected
> errors, especially given that we're making little/no effort to match
> error messages. would be good to have some kind of "i expect command c
> to fail with exit code x and stderr output that matches regex r"
> utility.

conference writeup: if I had a million dollars

Going deeper on that topic, here's David Wheeler's 009 dissertation:
Countering "trusting trust" through diverse double compiling.

https://dwheeler.com/trusting-trust/

That's _why_ reproducing builds from source is so important, and how it's just the _start_ of proper analysis.

Rob

P.S. If we had an unlimited budget I'd hire a couple recent graduates from a women's technical college to glue qemu's tcg to tinycc so we had a third compiler that could target superh, and then set them to making it reproduce https://bellard.org/tcc/tccboot.html with a current kernel+toybox+musl. And turn it into a multicall binary so ld/strip/nm/objdump aliases worked and it could replace binutils as well as cc...

Wikipdia:
  https://en.wikipedia.org/wiki/Tar_(computing)#Key_implementations

netcat udp mode netconsole?

mkroot
  export
  A || B
  if ! A
  then
  else
  fi
  $$
  "$(A)"
  && #
  ' '
  VAR="$(A)"
  exec
  /dev/"$A" $B

init script: turn verbosity _down_ so not crapping over prompt

toysh:
  when cursor up requires moving up, clear to end of line. (Clear to end of screen?)

FAQ:
  https://github.com/landley/toybox/issues/122#issuecomment-471052933

todo: 3 waves talk
  https://twitter.com/Iron_Spike/status/1104104745805975553

Rethinking PID 1
  http://0pointer.net/blog/projects/systemd.html

The tragedy of systemd
  https://www.youtube.com/watch?v=o_AIw9bGogo
  mount filesystems
  start daemons
  routed inetd telnetd cron getty
  service - persistent daemon, not called from inetd

  system configuration (mount filesystems, etc)
  service bootstrap (start/stop services)
    - container, security context

lunchd - service providing daemon manager
  - at, cron, init, inetd
  - listen on tcp, unix sockets
  - start on demand, restart
    - granular access to services (security)
  - listen to hardware and software changes
  - parallel

  kernel/system/user
    network manager
    udev

egrep "(set|put|unset|clear)env|environ" toys/*/*.c lib/*.c
  - new lib/env.c

[Side note: since bash also supports 'printf a%n b >/dev/null' as a
convoluted way of accomplishing 'b=1', I wonder if it would be
possible to port https://github.com/carlini/printf-tac-toe which
performs an interactive game of tic-tac-toe in a single printf(3)
statement invoked in a while loop into a single printf(1) command line
invocation. The lack of %hhd implicitly masking with 256 makes it
harder]


tilck, a tiny linux-compatible kernel that runs busybox
  https://github.com/vvaltchev/tilck

From: vladislav.valtchev at gmail.com (2/24/23 11:25)

Hello Rob.
There's no simple way to build Tilck without first running its 
build_toolchain script. Please, don't assume that the script is going to 
build a whole GCC toolchain from scratch: it's simply going to download 
a pre-built one (~60 MB from toolchains.bootlin.com), along with busybox 
etc. Here, "toolchain" doesn't mean just the GCC toolchain. It actually 
means all the packages that Tilck will use for compiling user space 
projects.

The whole project is designed to offer an impeccable user experience and 
be very fast to build the first time. Sorry that the README is not 
friendly to read with VIM, but that's how modern GitHub projects are. If 
I made it completely plain-text, it would be ugly on Github in the 
browser and many more people would complain about that. Therefore, 
please open the README.md in a browser and follow the instructions from 
there. Actually, the full instructions are here:

https://github.com/vvaltchev/tilck/blob/master/docs/building.md

The README contains just the basics to get the first build & run. In the 
building.md document, you'll find instructions about how to configure 
the project as well.


Also, there are no make files because the project uses CMake. CMake will 
generate the make files with the proper configuration. You'll be able to 
change such configuration and perform a light incremental build as well.


I hope this helps you. Let me know.

Regards,
Vlad




On 2/24/2023 6:37 PM, Rob Landley wrote:
> > On 1/10/23 14:00, Giovanni Lostumbo wrote:
>> >> Tilck, a Tiny Linux-Compatible Kernel
>> >> "It just runs mainstream Linux programs like the BusyBox suite."
>> >> https://github.com/vvaltchev/tilck
>> >> https://www.youtube.com/watch?v=Ce1pMlZO_mI
> > 
> > Finally got around to cloning the repo and trying this. The readme is basically
> > illegible in vi because it's got so much formatting it's not usable as a text
> > file, but I did catch it saying you just need i686-musl toolchain like you can
> > get from bootlin. I tried just typing "make" and it went:
> > 
> >    Makefile:37: *** Before building Tilck, you need to build the toolchain by
> >    running ./scripts/build_toolchain.  Stop.
> > 
> > And I went "no I don't, I've got like a dozen of them and the illegible README
> > just said I could get it from bootlin".
> > 
> > There's no "make help" output. No obvious way to tell it what toolchain to use?
> > The error message in the Makefile is generated from a lack of "$TCROOT" which is
> > set to "./toolchain2", but nothing in the top level Makefile actually uses the
> > variable or explains what should be in it?
> > 
> > I tried setting it to one of my existing toolchains, which broke with:
> > 
> >    $ TCROOT=~/mcm/ccc/i686-linux-musl-cross make
> >    ...
> >    CMake Error at CMakeLists.txt:118 (message):
> >      /home/landley/mcm/ccc/i686-linux-musl-cross/.gcc_tc_ver_i386 not found: run
> >      scripts/build_toolchain
> > 
> > I tried find . -iname '.gcc*' under i686-linux-musl-cross directory in case it
> > wanted a subdirectory of the compiler build output (I'd set CROSS_COMPILE to
> > ~/mcm/ccc/i686-linux-musl-cross/bin/i686-linux-musl-cross- for example) but
> > there were no files starting with .gcc anywhere other there.
> > 
> > I looked in its "kernel" subdirectory but there's no Makefile in there. It says
> > it's binary compatible with linux but there's no obvious way to provide it an
> > existing root filesystem...
> > 
> > That's about the attention I have for it right now. It was an interesting video,
> > but I can't get it to build.
> > 
> > Thanks,
> > 
> > Rob



#tar --transform
#  tests (in progress in ~/toybox/toybox)
#  --show-transformed-names
#sed --xform flags

why turtle no ctrl-c?
why turtle no ps?
Jeff wants old stupid /bin /usr/bin layout
  regression test squashfs
Jeff wants dynamic build
  - needs ldd selection
  - needs hardlinking


