Solving the mystery of hanging character set conversions in glibc's iconv utility

Website visitors don't typically consider character encoding and conversion when accessing digital content. However, engineers have been dealing with conversion issues since users started transferring strings from one computer to another. Today, when more than 95 percent of all web pages use UTF-8, converting data between character sets might seem less relevant. However, it continues to be useful when dealing with legacy data and systems, or with languages where multiple character sets are comparably popular.

For example, using EUC-KR is not unusual on Korean websites because it is more efficient for Korean text. A search engine that indexes Korean sites needs to be able to convert between EUC-KR and UTF-8 to perform the relevant string comparisons.

The iconv programming interface provided by POSIX, implemented by its namesake utility program, converts textual data from one character encoding to another. As an engineer on the Red Hat Platform Tools team, I was tasked with finding and fixing bugs reported in the GNU C Library (glibc) iconv utility. In this article, I discuss my experiences with fuzzing iconv and fixing bugs in the iconv front end.

Error handling in iconv

A typical iconv program invocation looks like so:

$ iconv -f ISO-8859-7 -t UTF-8

In this case, it is expected to read Latin and Greek text (encoded as ISO/IEC 8859-7) from standard input and write its UTF-8 equivalent to standard output.

While POSIX allows the iconv utility to exit with an error when given invalid input, the glibc iconv implementation has an extension that is more tolerant of erroneous input. The user can add a suffix to the destination character set specification to enable iconv features that can:

• Ignore invalid input characters while continuing to process further input.
• Transliterate input characters without an equivalent in the output character set to a sufficiently similar combination of output characters.

For example, if we wanted to convert UTF-8 encoded Czech text, which uses both diacritics and standard Latin letters, to ASCII, the following invocation would do the trick:

$ iconv -f UTF-8 -t ASCII//TRANSLIT

In this case, iconv transliterates the letters with diacritics into their diacritic-free equivalents.

String manipulation in iconv_open

This extension is probably implemented as a string suffix because the underlying iconv interface is a POSIX standard, and its signature doesn't allow for flags or options. Calls to iconv are made with a handler obtained by calling iconv_open, and with the appropriate conversion source and destination character encoding names. The iconv_open function has the following signature:

iconv_t iconv_open (const char *tocode, const char *fromcode);

Thus, the only meaningful way to extend its functionality without introducing a new interface is to pass option information in one of the string arguments.

The glibc iconv implementation recognizes and uses conversion source and destination encodings in the form of a slash-separated triplet:

• The first component of the triplet might be an encoding form, which implies a character set.
• Alternatively, it might be a character set where a second component is ideally an encoding form for the underlying character set. If the encoding form is left unspecified, iconv uses a default.
• The third component of the triplet is the optional suffix that enables GNU extensions. This means UCS-2// and ISO-10646/UCS-2/ are equivalent.

However, because most of the common iconv invocations simply use encoding forms, the second component of the slash-separated specification was forgotten when documenting the interface, and the manual page ended up with the following fragment:

Furthermore the GNU C library and the GNU libiconv library support the following 
two suffixes:

       //TRANSLIT
              When the string "//TRANSLIT" is appended to tocode,
              transliteration is activated.  This means that when a
              character cannot be represented in the target character
              set, it can be approximated through one or several
              similarly looking characters.

       //IGNORE
              When the string "//IGNORE" is appended to tocode,
              characters that cannot be represented in the target
              character set will be silently discarded.

There was no clear explanation of the triplet scheme and how multiple suffixes could be passed, which led to the implementation allowing calls such as:

$ iconv -f ISO-8859-7 -t UTF-8//TRANSLIT//IGNORE

Note that the missing second component and the resulting // were misunderstood as a suffix separator. Based on my reading, this misunderstanding seems to have touched the program code as well.

Debugging the iconv utility

Ideally, the iconv utility program would have avoided supporting options in the form of string suffixes. Instead, it might have supported them only by way of these options, but not in this case. It accepts the same suffixes as the underlying function.

In 2016, a bug was reported against the iconv utility, uncovering a hang when the TRANSLIT and IGNORE suffixes were used in conjunction with bad input and the -c option. This combination is meant to cause iconv to omit invalid characters in the input from the output, similar to IGNORE, as follows:

$ echo -en '\x80' | iconv -f us-ascii -t us-ascii//translit//ignore -c

The report gathered comments including hints at the possible causes and other similar hangs. I didn't make much progress on the report until 2019, when I started looking at it in depth.

Not knowing much about the internals of the iconv interface or utility, I decided to start with a bit of testing. I first wanted to check whether there were hangs in the iconv program other than the one reported. A "back of the envelope" calculation led me to conclude that it would take mere days to test every single two-byte input combination for all combinations of program switches and conversions—days that I wouldn't have to spend analyzing or debugging iconv code.

One bash script and roughly a week later, I presented the results. There were hangs while converting 165 character sets. Almost all, with the exception of four, occurred when using the option suffixes (TRANSLIT and IGNORE) and the -c switch, which silenced error messages upon encountering invalid characters. It seemed clear that the problem was general and did not correspond to a bug in a specific character set's converter.

After some debugging, I realized that the option suffixes were handled in string form during the program, instead of being used once to set corresponding flags that are subsequently used exclusively. Bugs in the code led to all but the first option being dropped. This led to hangs when the IGNORE option was passed after TRANSLIT.

Fixing this involved a somewhat significant change in iconv's option parsing logic, but once that was fixed, the hangs vanished.

Conclusion

Eventually, I fixed the remaining known hangs in iconv. I am finally beginning to feel a bit more comfortable with the codebase. There is still some work to do in glibc's iconv implementation—among other things, improving the manual page. A new interface would also be useful.

For more information on C and C++, please visit Red Hat's topic page.