Mangled Multibyte Characters

This forum is dedicated to the Aseco/Rasp 2 server records and control script by Flo and Assembler Maniac. Announcements and issues can be discussed in this forum.

Moderators: Flo, Assembler Maniac, f*ckfish, TM-Patrol

Post Reply
Hach-Que
cyclist
cyclist
Posts: 38
Joined: 19 Dec 2006 08:23
Owned TM-games: TMUF
Manialink(s): hach-que
Contact:

Mangled Multibyte Characters

Post by Hach-Que » 22 May 2009 12:56

I'm having trouble reading nicknames with unicode characters in them from the race results.

This is (part of) what the $results array contains onEndRace.

Code: Select all

    [0] => Array
        (
            [0] => Array
                (
                    [Login] => ayubu
                    [NickName] => $w$i$000Ãàm$ià¤$80f$wαγÏ
ÃÏ

                    [PlayerId] => 239
                    [Rank] => 1
                    [BestTime] => 119560
                    [BestCheckpoints] => Array
                        (
                            [0] => 36390
                            [1] => 49230
                            [2] => 54420
                            [3] => 102860
                            [4] => 118580
                            [5] => 119560
                        )

                    [Score] => 0
                    [NbrLapsFinished] => 0
                    [LadderScore] => 0
                )

            [1] => Array
                (
                    [Login] => hach-que
                    [NickName] => $000[V$F00i$000PER] $0ffĤÎĦΩÐЭ
                    [PlayerId] => 250
                    [Rank] => 2
                    [BestTime] => -1
                    [BestCheckpoints] => Array
                        (
                        )

                    [Score] => 0
                    [NbrLapsFinished] => 0
                    [LadderScore] => 0
                )

        )
ayubu is part of the Bam clan, but the system mangles it so that it is unreadable. Is there a way of reversing this, or preventing it in the first place?
Image

User avatar
oliverde8
solid chaser
solid chaser
Posts: 1135
Joined: 07 Oct 2006 07:03
Owned TM-games: TMU
Manialink(s): oliverde8
Location: Turkey
Contact:

Re: Mangled Multibyte Characters

Post by oliverde8 » 22 May 2009 15:50

thats what is shown in the cmd window, that doesen't support that chars

In code there are okay and readable you just can not show it on the terminal correctly

if you try to show a nickname in game maniaklink just use the htmlspecialchars and it will ok :thumbsup:
ImageImage
Manialink : oliverde8color>
Servers : STATS <> United | VeryShort | Made In Turkey

Hach-Que
cyclist
cyclist
Posts: 38
Joined: 19 Dec 2006 08:23
Owned TM-games: TMUF
Manialink(s): hach-que
Contact:

Re: Mangled Multibyte Characters

Post by Hach-Que » 23 May 2009 01:08

I can print the characters fine because I store the clans tags in an array and when I print_r them, the characters display fine. The console is capable of printing those characters, but they get mangled in the results.
Image

donaldl43
smooth traffic navigator
smooth traffic navigator
Posts: 194
Joined: 18 Jan 2009 21:07
Owned TM-games: TMU
Contact:

Re: Mangled Multibyte Characters

Post by donaldl43 » 23 May 2009 19:42

Im having a similar issue. I try to show those types of characters in my Simple Web Local records and they look strange. I've searched for help on getting the encoding correct but so far I've struck out.

Example:

This is what I see:
Image
And this is the text.
ミ.[CैИ].从エॆモイラモレシ

Those look identical to me. My browser is set to UTF-8 encoding.

User avatar
oliverde8
solid chaser
solid chaser
Posts: 1135
Joined: 07 Oct 2006 07:03
Owned TM-games: TMU
Manialink(s): oliverde8
Location: Turkey
Contact:

Re: Mangled Multibyte Characters

Post by oliverde8 » 23 May 2009 20:34

check encoding of the file when you save it.
ImageImage
Manialink : oliverde8color>
Servers : STATS <> United | VeryShort | Made In Turkey

donaldl43
smooth traffic navigator
smooth traffic navigator
Posts: 194
Joined: 18 Jan 2009 21:07
Owned TM-games: TMU
Contact:

Re: Mangled Multibyte Characters

Post by donaldl43 » 23 May 2009 20:59

oliverde8 wrote:check encoding of the file when you save it.
Well, the names come out of MySql databases. I have received a few database files from various people and they already look like that. They look like that if I view them in the Sql file and they look like that if I import them into a Sql database and view them in phpmyadmin.

Hach-Que
cyclist
cyclist
Posts: 38
Joined: 19 Dec 2006 08:23
Owned TM-games: TMUF
Manialink(s): hach-que
Contact:

Re: Mangled Multibyte Characters

Post by Hach-Que » 24 May 2009 01:04

Well, here's something very weird. I send the mangled data to a server file, which uses the same MySQL database as ASECO.

Obviously they don't match. Until I put:

Code: Select all

SET CHARSET utf8
SET NAMES utf8
at which point, my server file started retrieving the data from MySQL mangled in the exact same way! So if you need to compare nicknames with a MySQL field, instead of trying to unmangle the data, just get MySQL to mangle it by calling those queries above (it should be noted that ASECO has the exact same query calls, but I figure the nicknames are retrieved from TrackMania, so it can't be causing it for ASECO).
Image

donaldl43
smooth traffic navigator
smooth traffic navigator
Posts: 194
Joined: 18 Jan 2009 21:07
Owned TM-games: TMU
Contact:

Re: Mangled Multibyte Characters

Post by donaldl43 » 24 May 2009 01:19

I believe it has to do with the collation of the names in the MySql table. I finally figured out my issue was caused by some of the tables being utf8_general_ci, some being utf8_unicode_ci, and even others being some latin chars. I found some need the SET NAMES 'UTF-8' and others don't.

Post Reply