Hi Stan. There may be a bug, but it's unlikely.

When you run checksetup.pl without running recode, and it
converts your tables to UTF-8, all it does is take the exact bytes that
are in your system and make MySQL interpret them as UTF-8. It does not
do any translation or conversion. It deletes any data which is not
valid UTF-8.

When you enable the utf8 parameter in Bugzilla 3.0, it makes
Bugzilla's MySQL connection interpret data from the database as
UTF-8. It also sends data as UTF-8. This allows correct sorting and
searching for UTF-8 data.

Each column in every table has its own encoding. The default is
latin1. checksetup.pl converts each column from latin1 to binary to
utf8, thus preserving the *exact bytes* already in the database.

Thus, if the data was already in UTF-8 in your database (even
though the encoding says latin1), you wouldn't be having any problem,
if you just ran checksetup.pl.

Data which was not valid UTF-8, however, would have been

However, if the encoding was not latin1 in MySQL for your
2.20 database, and you used mysqldump to copy the database, that could
have corrupted your data in unforseeable ways. For example, it's
possible that your old MySQL had some cyrillic encoding (or was
actually in utf8) and you dumped it as latin1 (which is the default
for mysqldump) and then imported it as latin1 (which is the default on
*most* installations of mysql), in which case MySQL *does* actually
attempt to do some re-encoding itself, which can cause untold havoc.

There are a million possible combinations of things that could
have happened, but checksetup or recode being incorrect is not likely.
I myself tested recode on two common Russian charsets, and it worked
just fine.

If your data was already in utf8, there would be no need to run

Now that you have an understanding of the process, perhaps it
will be helpful to you in resolving this issue.

I am CC'ing this email to the support-bugzilla list, so that
others may benefit from it. If you have any further replies, make sure
you reply to the list, where many people can see and possibly advise
you on the situation.


On Thu, 12 Jul 2007 02:51:40 +0400 Stan
Grishin wrote:
> Max,
> well, there's a clearly bug in either of the checksetup.pl or
> recode.pl as neither supported upgrade from 2.20 with russian
> comments in the bugs as we have followed everything in the
> documentation. Thank you for your offer, but If we could have
> afforded the paid help for this, we would have probably gotten it
> when we noticed the problem.
> -Stan
> On 11-Jul-07, at 10:15 AM, Max Kanat-Alexander wrote:
> > Hey Stan. At this point I think the only way I can be of any
> > further assistance is as a contractor, looking directly at your
> > systems.
> >
> > -Max
> >


Competent, Friendly Bugzilla Services. And Everything Else, too.