[Toybox] [NEW TOY] iconv

[Felix Janda]

Rob Landley wrote:
> On 04/13/14 04:37, Felix Janda wrote:
> > Isaac Dunham wrote:
> > [..]
> >> locale and iconv were already triaged. 
> > [..]
> > 
> > iconv is actually something I'm missing on my current musl based system.
> > Attached is a simple version using the libc's iconv.
> 
> Small, simple command: lemme do the cleanup now so it doesn't build up.

Thanks.

> A global outside GLOBALS.
[..]

I also noticed later that void * for iconv_t should be fine.

[..]
> Really the only interesting errno case from iconv is illegal sequence.
> The rest just say "ran out of input" or "ran out of output" which is
> what you expect from a conversion that's not at the end of the file yet.
> (Ok, truncated sequence is a synonym for illegal sequence if we're not
> at the end of the buffer, which we can special case as at the _start_ of
> the buffer with the memmove logic.)

You mean "if we're at the end of the buffer"?

> Hmmm... we should probably pass illegal sequence bytes through. (Pass
> 'em through.) Except check if output buffer is full before doing that.
> (Don't have to check inleft nonzero because if inconv() returns illegal
> sequence but used up all the input buffer, that's a libc bug.)

Right...

> memmove() with length 0 isn't an error, is it? Ok.
> 
> Where would I get a test file to convert? I just ran a text file through
> it and confirmed it's not making any changes to it, but that doesn't
> mean much. :)

More interesting would be roundtrip encoding some files.
For testing, I just used an uim tarball[1] with some eucjp encoded files.
The cleaned up version still seems to work properly.
Even more interesting would be a file with some illegal sequences. I didn't
test that at all.

> (Sorry, rewrote it a bit more than I expected to. Checking in now...)

Error handling looks more sensible. Have you considered that iconv_open()
might also fail because of insufficient memory.

> P.S. Posix iconv has several more command line options. -c is easy and
> -s is NOP for us, but I dunno how to do -l.

glibc's doesn't have them. So I guessed that they are not much used.
Now I see that libiconv has them.


Felix

[1]: http://uim.googlecode.com/files/uim-1.8.0.tar.bz2

 1397427508.0