[Toybox] dd in toybox?

[scsijon]

On 09/07/19 20:31, Rob Landley wrote:
> 
> 
> On 7/9/19 2:55 AM, scsijon wrote:
>>
>>
>> On 09/07/19 16:56, Rob Landley wrote:
>>> On 7/8/19 10:41 PM, scsijon wrote:
>>>> I was wondering if you would consider adding parts of dcfldd
>>>> (https://linuxx.info/dcfldd/, https://linux.die.net/man/1/dcfldd and
>>>> http://dcfldd.sourceforge.net/) rather than having just the ordinary dd 'bits'
>>>> to your dd implementation if possible?
>>>
>>> What specific feature(s) are you asking for?
>>>
>>>> I know it's not new, but the extra
>>>> operands "pattern, textpattern and errlog" at least seem to be somewhat more
>>>> than just clearly usefull, with maybe the pair "hash and hashwindow" also if
>>>> possible.
>>>
>>> Um...
>>>
>>>> dcfldd is a newer version of dccifd, is functionally the same, but in parallel
>>>> able to calculate checksum using hashing algorithms are MD5, SHA-1 and SHA-2,
>>>> while it extends partially (by subscription).
>>>
>>> This differs from piping it through sha1sum how? Never having used this, it's
>>> not immediately obvious how or why to...
>>>
>>>> Or do you have some other way we can deal with these combined
>>>> functions easily?
>>>
>>> Pipelines? This is why Doug McIlroy suggested the pipe operator somewhere around
>>> 1973?
>>>
>>>     yes "hello" | dd
>>>
>>>> And as usual, i'll state that i'm not a coder nowadays, just a builder with a 3d
>>>> perspective outlook, I can't give you a patch, just an idea for someone to work
>>>> with. :-)}
>>>
>>> Could you give me some usage examples?
>>>
>>
>> ok, this is the usual example given for it,
>>
>> dcfldd if=/dev/source hash=md5,sha512 hashwindow=1G md5log=md5.txt
>> sha512log=sha512.txt \hashconv=after bs=512 conv=noerror,sync split=1G
>> splitformat=aa of=image.dd
> 
> tee </dev/source >(sha512sum>sha512.txt) >(md5sum>md5.txt) | split image.dd

OK, maybe i'm just use to doing it that way since I first came across 
dcfldd, if your way works and your happy with it, i'm not going to argue 
that another way is better as there is always more than one way to do 
something.

> 
>> and all done without external pipe, etc. overheads.
> 
> Is a pipe a significant overhead? (It naturally takes advantage of SMP, for one
> thing...)

Can be, i've come across iso's that fail to 'play nice' if smp dd'd but 
work ok if set to use a single processor
> 
>> My normal use though, is for fixing and changing formats in .iso  and .img
>> files, saves a total rebuild of an iso just to fix a typo, a bane in my life at
>> least (if not also for others) when a iso and img.gz are just about ready for a
>> release which is why I would like pattern and textpattern added.
> 
> Toybox sed should handle binary files?
> 
ok, haven't tried it for that, something to do when i've satisfactorily 
built my Quirky Thud to release as i've added toybox to that.

> Rob
> 
Anyway, regards and keep up the good work, you can take it that it's 
much appreciated by quite a few of us.

And as an OT; I wonder how we are going to take to 256bit, weve stepped 
from 4bit>8bit>16bit>32bit>64bit over the decades and now with the Ryzen 
going strong (out and available), there has been some talk on a few 
kernel (not just linux and BSD) related boards of it starting to be 
'formulated' as the largest of the Ryzen apparently can handle 'it' in 
some format or another. (Apparently 128bit is being skipped.) Sounds 
like the few mainframes still out there are due to dissapear in the next 
few years, and I wonder what's going to happen with the 'cloud' systems. 
Not sure I would want to be a coder then, it might get somewhat messy 
with instruction timing at least.