TMU Data Fetcher

This is the place where you can find everything related to the dedicated server, control scripts and community tools.

Moderators: Pit Crew, TM-Patrol

Xymph
Pit Crew
Pit Crew
Posts: 5648
Joined: 19 Aug 2007 12:58
Owned TM-games: TMN, TMU, TMF, TM²
Contact:

TMU Data Fetcher

Post by Xymph » 20 Jan 2008 15:24

The TMNDataFetcher class (originally by F*ckfish, and significantly updated by me since ASECO/RASP v0.92) provides an easy API to obtain lots of stats about a TMN login from Nadeo's master server. I figured it would be an interesting challenge to do the same for TMU logins.

And a challenge it was at first too, since packet sniffing TMU revealed that it uses an entirely different communication mechanism between client (or dedicated server) and the master server than TMN. While the latter employs no fewer than 7 different URLs from which to request various details via HTTP GET calls (easy to test in a browser), TMU uses one URL that exchanges XML blocks via HTTP POST calls (requires a bit of scripting to send/receive). Additionally, many types of XML requests to the master server require a valid session key to be included, and only a few return useful data without such a key (which an API cannot generate).

Fortunately those few requests include two to obtain basic player info and stats on a login, and two for traversing the player and zone ladders. So around those calls I've now developed a TMUDataFetcher class, providing an easy API for retrieving player and zone stats for a given TMU login. This includes that login's world stats, stats for all seven environments, and player ranking within each of his/her (sub)zones -- all of which can also been looked up at TM Ladder -- along with the rankings of each (sub)zone within its parent zone (just like what you see in the TMU client's server/zone browser).

Here is the return object's structure, using Mistral as a (hopefully willing :) ) example:

Code: Select all

TMUDataFetcher Object
(
    [version] => 0.2.1.0
    [zones] => 1
    [login] => mistral
    [nickname] => $FFFMi$88Fstr$FFFal
    [location] => World|Germany|Bavaria|Regensburg
    [worldrank] => 657
    [totalplayers] => 94308
    [points] => 11418.3
    [lastscore] => 0
    [wins] => 71134
    [losses] => 34774
    [draws] => 158864
    [envstats] => Array
        (
            [Bay] => Array
                (
                    [rank] => 437
                    [points] => 1531.96
                    [lastscore] => 2.51
                    [wins] => 9935
                    [draws] => 22519
                    [losses] => 3388
                )
            [Coast] => Array
                (
                    [rank] => 774
                    [points] => 1396.96
                    [lastscore] => 0
                    [wins] => 7587
                    [draws] => 15526
                    [losses] => 3962
                )
            [Desert] => Array
                (
                    [rank] => 601
                    [points] => 1399.1
                    [lastscore] => 0.14
                    [wins] => 9488
                    [draws] => 23588
                    [losses] => 5829
                )
            [Island] => Array
                (
                    [rank] => 496
                    [points] => 1927.04
                    [lastscore] => 3.33
                    [wins] => 11954
                    [draws] => 23331
                    [losses] => 4913
                )
            [Rally] => Array
                (
                    [rank] => 236
                    [points] => 1326.43
                    [lastscore] => 0.29
                    [wins] => 11500
                    [draws] => 17167
                    [losses] => 4749
                )
            [Snow] => Array
                (
                    [rank] => 396
                    [points] => 1573.94
                    [lastscore] => 2.01
                    [wins] => 11538
                    [draws] => 28763
                    [losses] => 6728
                )
            [Stadium] => Array
                (
                    [rank] => 1521
                    [points] => 3978.44
                    [lastscore] => 0
                    [wins] => 9240
                    [draws] => 28116
                    [losses] => 5271
                )
        )

    [zonestats] => Array
        (
            [World|Germany] => Array
                (
                    [zonerank] => 1
                    [totalzones] => 45
                    [zonepoints] => 167418
                    [playerrank] => 193
                    [totalplayers] => 18526
                    [playerpoints] => 11418
                )
            [World|Germany|Bavaria] => Array
                (
                    [zonerank] => 5
                    [totalzones] => 16
                    [zonepoints] => 124408
                    [playerrank] => 20
                    [totalplayers] => 2884
                    [playerpoints] => 11418
                )
            [World|Germany|Bavaria|Regensburg] => Array
                (
                    [zonerank] => 3
                    [totalzones] => 6
                    [zonepoints] => 70911
                    [playerrank] => 5
                    [totalplayers] => 69
                    [playerpoints] => 11418
                )
        )
)
Next I am going to develop a /statsall plugin for AsecoTMU with this. That also helps me to learn how to develop AsecoTMU plugins, which will be necessary for porting ASECO/RASP features to that framework, as AsecoTMU will be the starting point for getting things going on Forever.

Cheers! :)
Developer of XASECO for TMF/TMN ESWC & XASECO2 for TM²: see XAseco.org
Find your way around the Mania community from the TMN ESWC hub, TMF hub, TM² hub, and SM hub

m&o
happy cruiser
happy cruiser
Posts: 188
Joined: 19 Apr 2007 17:53
Owned TM-games: TMN, TMU, TMF

Re: TMU Data Fetcher

Post by m&o » 20 Jan 2008 21:54

looks like you uploaded the tmn datafetcher to your site

EDIT
looks like your url is wrong the tmudatafetcher link sends you to the tmndatafetcher download

(for other people use this till he updated http://www.gamers.org/tmn/tmudatafetcher.zip )
Image

Xymph
Pit Crew
Pit Crew
Posts: 5648
Joined: 19 Aug 2007 12:58
Owned TM-games: TMN, TMU, TMF, TM²
Contact:

Re: TMU Data Fetcher

Post by Xymph » 20 Jan 2008 22:54

m&o wrote:looks like you uploaded the tmn datafetcher to your site

EDIT
looks like your url is wrong the tmudatafetcher link sends you to the tmndatafetcher download

(for other people use this till he updated http://www.gamers.org/tmn/tmudatafetcher.zip )
Oops, that's #37 in the list of ways copy & paste can break things for ya. :oops: Thanks, it's fixed now.
Developer of XASECO for TMF/TMN ESWC & XASECO2 for TM²: see XAseco.org
Find your way around the Mania community from the TMN ESWC hub, TMF hub, TM² hub, and SM hub

smurf
happy cruiser
happy cruiser
Posts: 116
Joined: 26 Sep 2006 22:15
Owned TM-games: TMNF TMUF
Manialink(s): ladder example
Location: France
Contact:

Re: TMU Data Fetcher

Post by smurf » 21 Jan 2008 06:51

Hi,
Good script :) , but I think, (sub)zones rankings is the worst idea I see :!: :cry: . If I don't display it on tm-ladder.com there are some reasons...
I explain:
I add a request counter to your script and results are really frightening:
- For a top player (top 20; i tried keter) your script send 9 requests to Nadeo's servers.
- For a middle level (like me schtroumf) your script send near 30 requests.
- For a bad player (i tried _fffsofiane), your script send 340 requests before my web server timeout !!!!
Giving this script for everyone is dangerous for Nadeo's servers and may cause lags on its...
Image

User avatar
f*ckfish
Pit Crew
Pit Crew
Posts: 1302
Joined: 15 Jun 2007 07:59
Owned TM-games: TMNF, TMUF
Location: Dresden, Germany
Contact:

Re: TMU Data Fetcher

Post by f*ckfish » 21 Jan 2008 13:13

Thanks a lot for the script. Although I don't own TMU it always bugged me how to do the fsock request^^
inactive, sorry

Xymph
Pit Crew
Pit Crew
Posts: 5648
Joined: 19 Aug 2007 12:58
Owned TM-games: TMN, TMU, TMF, TM²
Contact:

Re: TMU Data Fetcher

Post by Xymph » 21 Jan 2008 13:55

smurf wrote:Good script :) , but I think, (sub)zones rankings is the worst idea I see :!: :cry: . If I don't display it on tm-ladder.com there are some reasons...
I explain:
I add a request counter to your script and results are really frightening:
- For a top player (top 20; i tried keter) your script send 9 requests to Nadeo's servers.
- For a middle level (like me schtroumf) your script send near 30 requests.
- For a bad player (i tried _fffsofiane), your script send 340 requests before my web server timeout !!!!
Giving this script for everyone is dangerous for Nadeo's servers and may cause lags on its...
Well, first of all it's not that likely that many logins are being searched simultaneously using this script, and Nadeo's server seems pretty beefy as it hosts the TMS, TMN and TMU master servers all on the same IP. But you're right that the number of requests can be rather high depending on the login's rank and the number of players in the zone. This problem is most likely to occur for French and German players, as 21100+ out of the total 94400+ TMU logins (or 22 %) are in France, closely followed by 18500+ German ones (19 %). :)

There is a way to obtain the (sub)zone numbers without traversing the ladders, but like I wrote above that is one of those things that requires a valid session key, which was rather disappointing to discover. So good or bad, searching the zone ladders was the only idea that worked. All I could do is to make it an option, and make the searching process as efficient as possible (e.g. by requesting 40 entries at a time instead of the default 20).

However, you made me think about it a bit more, and I added two optimizations: first, all main requests for info & world stats are now bundled into one XML block, reducing the number of requests to the master server from 3 to 1 if the $zoneInfo option is false. And second, I added a delimiter to simply stop searching the zone ladders if the player is not found in the top-4000 (100 requests). You can further reduce that delimiter as you like of course, and the player rank for that zone is returned as -1. Your examples then take 7 requests (keter), 25 (schtroumf), and 199 (_fffsofiane) with no timeout anymore.

Download TMUDataFetcher v1.2 from the usual place.
Developer of XASECO for TMF/TMN ESWC & XASECO2 for TM²: see XAseco.org
Find your way around the Mania community from the TMN ESWC hub, TMF hub, TM² hub, and SM hub

mingger
highway camper
highway camper
Posts: 4
Joined: 26 Dec 2007 14:31
Owned TM-games: united

Re: TMU Data Fetcher

Post by mingger » 22 Jan 2008 17:00

hi one there will make a homesite to me (free)

User avatar
f*ckfish
Pit Crew
Pit Crew
Posts: 1302
Joined: 15 Jun 2007 07:59
Owned TM-games: TMNF, TMUF
Location: Dresden, Germany
Contact:

Re: TMU Data Fetcher

Post by f*ckfish » 22 Jan 2008 18:28

mingger wrote:hi one there will make a homesite to me (free)
Well... erm... No, I don't think so... Maybe if you specify your needs we can give you some hints but not like this ;-) BTW your request seems a bit Off-Topic.
inactive, sorry

Fatmax
pedestrian
pedestrian
Posts: 5
Joined: 29 Feb 2008 10:04
Owned TM-games: TMN

Re: TMU Data Fetcher

Post by Fatmax » 29 Feb 2008 10:09

Hello!

I downloaded the TMNDatafetcher und uploaded it to my webspace.

My question is, where do i have to write my LoginName in the script that i can see my stats.
PHP is running at the server, because i use some other php-scripts and they are working!

I hope somebody can help me!

Thanks!

Fatmax

Xymph
Pit Crew
Pit Crew
Posts: 5648
Joined: 19 Aug 2007 12:58
Owned TM-games: TMN, TMU, TMF, TM²
Contact:

Re: TMU Data Fetcher

Post by Xymph » 29 Feb 2008 19:46

Fatmax wrote: I downloaded the TMNDatafetcher und uploaded it to my webspace.

My question is, where do i have to write my LoginName in the script that i can see my stats.
PHP is running at the server, because i use some other php-scripts and they are working!

I hope somebody can help me!
Look at the tiny tmndatafetch.php script included with the download, that shows how to call the TMNDatafetcher with what parameters.
Developer of XASECO for TMF/TMN ESWC & XASECO2 for TM²: see XAseco.org
Find your way around the Mania community from the TMN ESWC hub, TMF hub, TM² hub, and SM hub

Fatmax
pedestrian
pedestrian
Posts: 5
Joined: 29 Feb 2008 10:04
Owned TM-games: TMN

Re: TMU Data Fetcher

Post by Fatmax » 01 Mar 2008 10:32

It doesnt't work!

Code: Select all

#!/usr/local/bin/php -q
<?php
// vim: set noexpandtab tabstop=2 softtabstop=2 shiftwidth=2:

// Simple command line driver for TMNDataFetcher class
// Created Jan 2008 by Xymph <tmn@gamers.org>

	include_once('tmndatafetcher.inc.php');

	$login = $argv[1];
	$data = new TMNDataFetcher($login, true);

	print_r($data);
?>
I changed

Code: Select all

$login = $argv[1];
to

Code: Select all

$login = $argv[Fatmax];
but nothing happens.

what about

Code: Select all

#!/usr/local/bin/php -q
?
Do i also have to change this? If yes, i dont' know what to write there!

Has somebody running this script?

Thanks!

Fatmax

User avatar
f*ckfish
Pit Crew
Pit Crew
Posts: 1302
Joined: 15 Jun 2007 07:59
Owned TM-games: TMNF, TMUF
Location: Dresden, Germany
Contact:

Re: TMU Data Fetcher

Post by f*ckfish » 01 Mar 2008 12:01

Try

Code: Select all

$login = 'Fatmax';
;-) Did you code php before?
inactive, sorry

Fatmax
pedestrian
pedestrian
Posts: 5
Joined: 29 Feb 2008 10:04
Owned TM-games: TMN

Re: TMU Data Fetcher

Post by Fatmax » 01 Mar 2008 12:15

Hello!

Thanks for the answer!

No, i didn't code php before.

I used only some ready php-scripts on my webpage, but there were only some little things to change like username....
In these scripts there were good comments so i did know what is necessary to change.

@f*ckfish: is this script running at you?

I tried your tip but it doesn't show anything!

User avatar
f*ckfish
Pit Crew
Pit Crew
Posts: 1302
Joined: 15 Jun 2007 07:59
Owned TM-games: TMNF, TMUF
Location: Dresden, Germany
Contact:

Re: TMU Data Fetcher

Post by f*ckfish » 01 Mar 2008 12:29

Yepp, totally works for me. But to be honest, I don't think you'll be able to handle this script without at least some basic php knowledge. You better try out some tutorials first.
inactive, sorry

Fatmax
pedestrian
pedestrian
Posts: 5
Joined: 29 Feb 2008 10:04
Owned TM-games: TMN

Re: TMU Data Fetcher

Post by Fatmax » 01 Mar 2008 13:04

Now I downloaded the script from your website and i will try it with this script.

It would help if you tell me in which file i have to change something.

Please!

Thanks!

Fatmax

Post Reply