[Toybox] [PATCH] blkdiscard (for review)

[Rob Landley]

On 4/14/20 6:36 PM, Patrick Oppenlander wrote:
>>> Without FDPIC every
>>> toybox fork/exec requires a another copy of the entire program image
>>> which kinda sucks.
>>
>> Run "make change", then look in the "change" directory.
>>
>> (It's "change for a $20". There should probably be be a better name for it.)
> 
> What's the use case for the separate binaries?

Mostly that people complain about not having it, but on a nommu system without
fdpic or binflt (which in my case means static PIE) you load less in for each
binary that can't share anything with any other binary.

> I thought the point of a single multi-call was that the total is less
> than the sum of the parts?

It is. But it creeps some people out who go "that's not how gnu does it". (But
it IS how gzip's been doing it forever...)

>>> Then again, FDPIC for ARM should be available in gcc 10 (& musl
>>> hopefully soon after) which ought to cover the majority of that space.
>>
>> Oh hey. Those french guys finally got their stuff pushed upstream? (I poked Rich
>> about that 4 years ago, https://www.openwall.com/lists/musl/2016/12/13/2 but he
>> decided to wait for it to wind up in the upstream compiler before caring. He has
>> a contract to maintain the fdpic stuff for sh2 and had the concepts explained to
>> him by Jeff Dionne the original author of uClinux, so the base support is there.
>> But there wasn't much he could do about the compiler...)
> 
> Yep, looks like it went in September 2019.
> 
> FDPIC for ARM is also on the musl 1.2.2 roadmap
> https://wiki.musl-libc.org/roadmap.html aimed for June release.

Cool. For cortex-m scripts/mcm-buildall.sh is doing
"armv7m:eabi:--with-arch=armv7-m --with-mode=thumb --disable-libatomic
--enable-default-pie" which provides a compiler I can then "make root
CROSS=armv7m" which makes static pie binaries that qemu-arm -cpu cortex-m3 can
sort of run?

$ qemu-arm -cpu cortex-m3 root/armv7m/fs/bin/toybox echo hello world
hello world

But not reliably:

$ qemu-arm -cpu cortex-m3 root/armv7m/fs/bin/toybox cal
qemu: uncaught target signal 4 (Illegal instruction) - core dumped
Illegal instruction

(Without the -cpu it runs fine. Something toolchainish I suspect? I don't THINK
an unaligned access issue would cause illegal instruction and sh4 should catch
that anyway.)

But the main reason I haven't put more work into that target is I haven't got a
kernel config and qemu-system-arm -M cortex-m-board pair I can use yet. The
cortex-m board I (briefly) had to run Linux on was an expensive FPGA thing
Innoflight loaned me for a bit, but took back again last year. (It had a
cortex-m bolted to the side of the FPGA. The kernel for that was an out-of-tree
fork maintained by Emcraft in Moscow, and all the make root tagets are vanilla
kernel with no patches.)

>>> So arguably simple is best. Would you like me to change it?
>>
>> I'm already fiddling with it here, but... hmmm.
> 
> I sent a simple patch to clean up the error return paths before you
> sent this. Feel free to ignore it and do your magic :)

It's in the pile, I'm just trying to finish the shell and cut a release...

>> I have automatic atolx parsing in optstr with # instead of : BUT it stores the
>> result in a long which means on 32 bit systems them maximum value is 2 gigs, and
>> this combines badly with large file support.
>>
>> On 64 bit systems, # is obviously the right thing to do. But when supporting 32
>> bit systems with large block devices, what you're doing (save the string then
>> parse in main) is the right thing to do.
> 
> I saw the # support in optstr. That reminds me -- I made a note that
> the Toybox M/MB/MiB parsing doesn't match blkdiscard where MB is *1000
> and M, MiB are * 1024. I overlooked that you have the 'd' support for
> decimal sizes so I guess that's intentional and consistent across all
> commands which take size arguments?

  toybox --help
  ...
  Numerical arguments accept a single letter suffix for
  kilo, mega, giga, tera, peta, and exabytes, plus an additional
  "d" to indicate decimal 1000's instead of 1024.

> Supporting powers of 1000 here is useless anyway. The offset & length
> need to be aligned to the underlying device block size, which, for all
> devices I know of is a power of 2.

Ok.

> That's another confusing thing with blkdiscard. The official
> documentation says "The provided value will be aligned to the device
> sector size." which I interpreted as meaning that the code would align
> an offset of 99 to something sane. But, do you round down or up? So I
> tried the one installed on my Arch Linux system and:
> 
> % blkdiscard -o 99 /dev/sdc
> blkdiscard: /dev/sdc: offset 99 is not aligned to sector size 512

The man page says Lukas Czerner ⟨lczerner at redhat.com⟩ maintains it, so I guess
ask him which way to go? (I'd say "pick one" but if this is a security thing you
 want to err on the side of removing data, and if it isn't you'd want to err on
the side of KEEPING data, and I have no idea?)

If he doesn't reply, I guess err on the side of removing data? If you were told
to discard data in this block, that means discarding the whole block. Otherwise
blkdiscard with a 1 byte range is a NOP...

> I didn't bother with testing for that case as the kernel spits the
> dummy if you give it stupid numbers anyway, so I didn't see the point
> of the extra ioctl to find the block size and the extra code to
> duplicate tests the kernel does anyway.

Also a valid approach. Possibly the fix that needs to go into Lukas' version is
a change to the man page to stop claiming it does something it doesn't. :)

>> And so on, and so forth... Eh, I'm sure it'll be fine. Never caused a problem
>> before. That we know about. (In case you're wondering why my $DAYJOB is with
>> people who made their own processor from scratch... There's reasons for it.)
> 
> I love the idea of J-Core. Is there readily available silicon now or
> do you still run on FPGA?

We still run on an FPGA. We met with some chip fab guys from Taiwan the
_morning_ we flew out of Japan due to Coronavirus back in February. (They sat 6
feet away from us. I mentioned it briefly in
https://landley.net/notes-2020.html#18-02-2020 .) Right now we're just trying to
survive the pandemic.

On the bright side, the ICE40 version now builds a working bitstream with an
entirely open source toolchain (the
https://lists.j-core.org/pipermail/j-core/2019-November/000868.html guys finally
got enough bugs fixed, although I need to finish unifying the j1 and j2 trees
and push that to github, what's up there now still only builds with the lattice
tools because we had to rewrite the register file to get the new toolchain to
recognize it's a RAM and not a pile of flip-flops. New one's a lot cleaner
anyway, but the commit's not against the old tree...)

>> Me, I'm a software guy. I'm 2700 lines into writing a hopefully 3500 line bash
>> replacement, months late on cutting a release, and feeling REALLY GUILTY I
>> haven't written a status update for my 11 supporters on
>> https://patreon.com/landley but I'm trying to get to a checkpoint I can report
>> as progress. I've ALMOST got "make root" booting to a shell prompt through the
>> init script...
> 
> I know words of encouragement are less value than Patreon $, but,

Oh, I don't _expect_ anyone to donate. I hugely appreciate when they do, but my
mortgage alone is $2k/month. It would be LOVELY if patreon replaced my $DAYJOB,
but the math's against it happening any time soon.

Still, I should remember to tell people it exists. Maybe someday...

> thanks for all your hard work!

I'm glad you like it. :)

> Patrick

Rob