[Toybox] xargs "argument too long"

[Rob Landley]

On 7/8/19 4:06 PM, enh wrote:> patch attached.
> i did check, in case we're crazy, and the kernel does do the right
> thing, there's no weird special case for these bytes in the kernel
> (this is linux; i didn't actually check a BSD kernel):
>

The kernel changed how this worked _twice_ since I wrote xargs.

It _was_ 32 pages, hard limit. I determined it experimentally on my ubuntu box,
tracked down the relevant kernel code to examine it, and implemented measurement
for that:

https://landley.net/notes-2011.html#17-12-2011

Then the kernel changed it to "2 billion envp[] length plus 2 billion argv[]
length, each individual string 2 billion long", and I recorded the commit on the
design.html page
(https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b6a2fea39318).

Then they arbitrarily retrofit a 10 meg limit on, and I pulled you into my
conversation with Linus on the inability to query the size and request to please
stop changing it, and I became Sad about this topic.

> /*
> * We must account for the size of all the argv and envp pointers to
> * the argv and envp strings, since they will also take up space in
> * the stack. They aren't stored until much later when we can't
> * signal to the parent that the child has run out of stack space.
> * Instead, calculate it here so it's possible to fail gracefully.
> */
> ptr_size = (bprm->argc + bprm->envc) * sizeof(void *);
> if (limit <= ptr_size)
> return -E2BIG;

I don't think it's on the stack in Linux. It's its own allocation passed into
the executable via the ELF auxiliary vector? (Did I ever tell you about the time
I worked out how to stick printfs into the libc dynamic library loader to debug
why it wasn't relocating _itself_ properly on the hexagon port? It's in my blog.
Second half of 2010 somewhere...)

> oddly the [Free]BSD xargs does count the space for the pointers for
> the common arguments given on the xargs command line, but not the
> pointers for the "extra" arguments from the input.

https://www.youtube.com/watch?v=2t-hyB8ibgk

Rob