[Toybox] dd tests for transaction size?

[enh]

On Sun, Jul 9, 2017 at 4:23 PM, Rob Landley <rob at landley.net> wrote:
>
> On 07/09/2017 05:18 PM, enh wrote:
>> >     On 07/09/2017 02:41 AM, Rob Landley wrote:
>> >     > does anybody have a decent strategy for testing the ibs
>> >     > and obs options?
>> >
>> >     Has anybody actually used the conv=sync option in the past 20 years? It
>> >     doesn't do what you think (that's conv=fsync), instead it pads short
>> >     reads with zeroes so the input block size is always the same.
>> >
>> >     How is that useful?
>>
>> It's used in various boot disk generation scripts in the Android tree.
>> (Whether it's needed is a question I can't answer as easily!)
>
> Hang on, do you use sync= or fsync=?

the only code owned my either of _my_ teams uses conv=fsync. but a
search for conv=sync turned up _more_ references to conv=sync than
conv=fsync.

> Because sync= pads _all_ short reads to block size with zeroes, not just
> the last one. I can see this being used to pad the _last_ block with
> zeroes, but it sounds like it could also corrupt the data if there are
> short reads before that.

the users that had comments said they wanted the last block padded.
padding all short reads (including EINTR ones) seems insane.

> I say this because SIGSTOP/SIGCONT used to cause short reads, and the
> interrupted read returned the data it had gotten when you continued.
> (Yes, I had builds break because I suspended and resumed them, and root
> caused it.)
>
> The really annoying part was it would cause _zero_ length reads which
> didn't mean end of file, you had to check errno for EAGAIN to
> distinguish that. I think the kernel guys fixed it so it won't do that,
> but if you SIGSTOP something that's doing a 1G single read() and then
> SIGCONT what happens these days?
>
> Hmmm...
>
>   $ cat > richards.c << EOF
>   #include <stdio.h>
>   #include <stdlib.h>
>
>   int main(int argc, char *argv[])
>   {
>     printf("len=%d\n", read(0, malloc(1000000000), 1000000000));
>   }
>   EOF
>   $ gcc richards.c
>   $ ./a.out < coderush.mp4
>   ^Z
>   [1]+  Stopped
>   $ fg
>   len=318818698
>
> Oh that's just spectacular. The SIGSTOP doesn't take effect until the
> read system call returns, even if it's reading 300 megs from rotating
> media. Thank you SO much linux guys. (Having done a couple runs with
> drop_caches I can confirm that ctrl-C doesn't work until the syscall
> returns either. That's just beautiful. Obviously that's never going to
> cause a problem ever, driving a system deep into swap thrashing... Reads
> from local disk block in D state now?)
>
>   $ ./a.out < /dev/zero
>   ^Z
>   [1]+  Stopped
>   landley at driftwood:~$ fg
>   ./a.out < /dev/zero
>   len=96399360
>
> Right, at least THAT still behaves like I expected. So SOMETIMES reads
> will be shortened by suspend/resume cycles. And other times they block
> in D state. (Yay inconsistent behavior!)
>
> Sigh. I suppose android builds are always happening from local disk (not
> network filesystems) and are never suspended during a build, so you
> don't need to worry about it?

i suspect that's why no-one notices the insanity. (when we first
started using the gold linker elsewhere in google i had to fix its
EINTR behavior. happens all the time on a FUSE file system, but folks
had been using it for years on regular file systems without noticing.)

> Rob



-- 
Elliott Hughes - http://who/enh - http://jessies.org/~enh/
Android native code/tools questions? Mail me/drop by/add me as a reviewer.