Keith's Weblog: Comments on "A regular expression to check for valid UTF-8"

by Aidan Kehoe

Wed, 30 Jun 2004 13:02:37 +0000

if (@iconv($input, 'UTF-8', 'UTF-8') == $input) echo "Good UTF-8!";

else echo "Nope."

is much, much faster, if iconv() is available.

by Micah

Wed, 30 Jun 2004 15:34:12 +0000

More regexps for Asian character-set encodings in this paper:

http://examples.oreilly.com/cjkvinfo/perl/svpm99-paper.pdf

by Rodent of Unusual Size

Thu, 19 Aug 2004 15:59:47 +0000

actually, after looking at rfc3629 and talking to sam ruby, it appears that this regex is, in fact, not valid. the rfc specifies that octets 0x00-0x7F are valid, being identical to their ASCII counterparts, but the regex excludes much of that range. it looks as though the utf-8 definition was conflated with the xml 1.0 character list to the detriment of the stated purpose ('is this valid utf-8?' determination).

by Keith

Thu, 19 Aug 2004 16:20:00 +0000

Thanks for pointing that out Ken! I'd wondered about that myself but didn't think too much about it. You're probably right that they conflated the XML 1.0 character definition with the definition of UTF-8.

by Andy

Tue, 20 May 2008 21:36:41 +0000

Quick Q if someone can tell me. Is Â valid utf-8 or must it be   ? Trying to insert XML (unencoded) into a DB that was reportedly set for UTF-8. It is failing and I am trying to understand if I need to encode or the DBA should fix the DB.

by Keith

Wed, 21 May 2008 04:13:06 +0000

Run the above regular expression against your XML to see if the Â is actually encoded in UTF-8 or whether it's Latin-1 or something. If the db is supposed to accept UTF-8 and your XML passes the regex, then the db is wrong. More likely, your XML isn't actually encoded in UTF-8.

by Al

Thu, 25 Jun 2009 19:29:22 +0000

"Confalted" -- cool learned a new word
Actually came for the regex(very useful thanks!)