php5.6/php7 compatibility + memleak fix

[Bueddl]

Hey everyone,

as some of you already noticed, there are some bigger issues for running some of the latest controllers on php 5.6/7.x. Mainly there are three tasks to perform in order to prepare the controllers to run on php 7.

1. Mysql deprecation

There is already a post [1] outside covering this for xaseco. It advertises a small plugin coded by reaby that basically does a wrap-up of the obsoleted mysql API and internally uses the mysqli api. I am not aware of the state of other controllers than xaseco and manialive here. Although this solution might work for you, there is a better way, which includes a little more effort. However, if you are just for an "it just works"-solution, you might be happy with it.

Anyway. it is only about replacing the old api. I always prefer using the object-orientated style. I did simply use some regex-enabled code editor and did a quick search and replace on those api parts. This worked for me very well, but keep in mind, that unless the plugin author will use the mysqli api in further releases, you would have to adapt this every time. So maybe, this is not the best solution for you. If you wanna go this way to hell, you can use these regexes for search-n-replace (each first line is search query, next is replacement):

Code: Select all

mysql_(query|ping|select_db)\((.*?)\)
\$aseco->db->$1($2)

mysql_(fetch_array|fetch_object|fetch_assoc|fetch_row)\((.*?)\)
$2->$1()

mysql_free_result\((.*?)\)
$1->free()

mysql_num_rows\((.*?)\)
$1->num_rows

mysql_(affected_rows|error|errno)\(\)
\$aseco->db->$1

Manialive and dependent projects are by the way not affected by this, as the developers of NADEO did carefully write good and fresh code here (good job this time guys ).

2. Memory leaks

Even if you got your controller running with the patch of reaby listed above, it won't run for long as there are several memory leaks present in the current versions of xaseco (1.16), fast (3.2.3t/4.0.0t) and probably other server controllers as well. All of them reside in the fact, that the resource handling was changed to a more strict one beginning with PHP7.0.
For those controllers, this means the usage of the xml_parser component inside three major components:

• Parsing incoming XMLRPC/GbxRemote messages from the gameserver
• Parsing the mapfile metadata
• Parsing various configuration files (not the variant that uses simplexml_load_xy)

The last one is not that big deal as it isn't a recurring task, the other two are more of interest. Anyway, fixing those issues is not a big deal, as it only requires explicitly unsetting the resources after invoking xml_parser_free. You can find a set of replacement files on the site linked below:
https://ftp.bueddl.de/tm/php7_patches/

Manialive isn't affected again, as they did use the simplexml_load_string function, which does not use a resource and therefore is not affected.

3. Even more deprecation

There are a few more deprecated functions, here is a list of obsoleted functions and their possible replacement:

• split() -> explode()
• If I remember more, I will update this post

So, there is a little work to do for gaining full compatibility, but if you stick to reabys plugin and my patched core files, you should get a more-less working controller running on PHP5.6/7.x

Cheers,

Bueddl


[1]: "modernizer" plugin by reaby for mysql/mysqli API convergence - viewtopic.php?f=127&t=35817
[2]: patched files including the memory leak fix - https://ftp.bueddl.de/tm/php7_patches/

[Xymph]

Thanks for the detailed post. I've added the unsets to my sources -- though as before a future release of XAseco1 and/or 2 is most unlikely.

Your zips contain GBXDataFetcher v2.5. Please don't redistribute ancient versions, the current one was v2.8.
And the three files should be identical between X1 and X2 anyway, yet yours are different in code and formatting. Strange.

In xmlparser.inc.php you changed closeTag() line "$this->struct[$to][$from][] = $this->struct[$from];" into:

Code: Select all

            $top = $this->struct[$from];
            if (!is_array($this->struct[$to]))
                $this->struct[$to] = [];
            if (!is_array($this->struct[$to][$from]))
                $this->struct[$to][$from] = [];
            $this->struct[$to][$from][] = $top;

Could you explain why?

[undef.de]

In my downloaded version is the line at closeTag() exact like before:

Code: Select all

$this->struct[$to][$from][] = $this->struct[$from];

Btw.: Thanks Bueddl, i've added this memleak fixes to UASECO too, everything else was already changed.

[Bueddl]

Hey Guys

Thanks for the detailed post. I've added the unsets to my sources -- though as before a future release of XAseco1 and/or 2 is most unlikely.

I was hoping that the original distributors would include these changes as it would make things easier if people won't have to modify their versions - just to mention further update compatibility (if there are any) and so on. This is why I am "selling" it as a hotfix and nothing more
Howevery, thanks for mentoining, I will update later this day.

Your zips contain GBXDataFetcher v2.5. Please don't redistribute ancient versions, the current one was v2.8.
And the three files should be identical between X1 and X2 anyway, yet yours are different in code and formatting. Strange.

I missed that one. I took the versions from the xaseco.org website a few days ago. The formatting seems to be from my IDE (sorry).

In xmlparser.inc.php you changed closeTag() line "$this->struct[$to][$from][] = $this->struct[$from];" into:

Code: Select all

            $top = $this->struct[$from];
            if (!is_array($this->struct[$to]))
                $this->struct[$to] = [];
            if (!is_array($this->struct[$to][$from]))
                $this->struct[$to][$from] = [];
            $this->struct[$to][$from][] = $top;

Could you explain why?

In my downloaded version is the line at closeTag() exact like before:

Code: Select all

$this->struct[$to][$from][] = $this->struct[$from];

The changes in the version xymph quoted is required for PHP 7.1 support.
The xml parser class used to do it the way undef quoted, but it isn't any longer valid to specify more than one previously undefined array depth level at once (so this is when we add the first child to $this->struct[$to][$from] = ...).

The xml parser works using a stack and build up the documnt structure in the $this->struct member variable. It is using a stack - as already mentoined - and when completing a xml tag, it removed the current element from stack and wraps the previously (therefore inner, called $from) xml tag into the now closed tag (called $to). This wrap up is illegal for PHP 7.1.
As the $this->struct[$to][$from] tag is still mutating, so adding childs (current $top might not be the only one), we need to go through this step by step.
I do this by backupping the $top of the stack, which is intended to be wrapped, and then go along the new location level by level to check if the array needs to be created (== $top is the first child). At the end I am appending the new top element.

There is propably (very likely) a more elegant way to do this, but this worked for me and is technically correct, so I was fine with it.

I am aware, that using [] for initializing a new array is breaking pre PHP 5.4 compatibility, so maybe this sould be replaced by array() in official versions to support a wider range of environments

Btw.: Thanks Bueddl, i've added this memleak fixes to UASECO too, everything else was already changed.

Nice!

Cheers,

Bueddl

[Kripke]

Thx bueddl!

but my system is bulling me

I get always php errors. Maybe my system is just retarded, I don't know.

Code: Select all

End Round
End Challenge
[PHP Warning] A non-numeric value encountered on line 1547 in file /gameserver/TM1/Killer/TMNF/xaseco/aseco.php
[PHP Warning] A non-numeric value encountered on line 1549 in file /gameserver/TM1/Killer/TMNF/xaseco/aseco.php
[02/04,16:18:42] kripke won for the 2. time!
[02/04,16:18:43] [RASP] Calculating ranks...
PHP Fatal error:  Uncaught ArgumentCountError: Too few arguments to function mysql_insert_id(), 0 passed in /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php on line 252 and exactly 1 expected in /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.compatibledatabase.php:104
Stack trace:
#0 /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php(252): mysql_insert_id()
#1 /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php(306): Rasp->getChallenges()
#2 /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php(618): Rasp->resetRanks()
#3 /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php(633): Rasp->onEndrace(Object(Aseco), Array)
#4 /gameserver/TM1/Killer/TMNF/xaseco/aseco.php(1249): event_endrace(Object(Aseco), Array)
#5 /gameserver/TM1/Killer/TMNF/xaseco/aseco.php(1602): Aseco->releaseEvent('onEndRace', Array)
#6 /gameserver/TM1/Killer/TMNF/xaseco/aseco.php(1090): Aseco->endRace(Array)
#7 /gameserver/TM1/Killer/TMNF/xaseco/aseco.php(760): Aseco->executeCallbacks()
#8 /gameserver/TM1/Killer/TMNF/xase in /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.compatibledatabase.php on line 104

(same error with the modernizer on other servers.

and this

Code: Select all

[PHP Warning] A non-numeric value encountered on line 1490 in file /gameserver/TM1/Killer/TMNF/xaseco/aseco.php
[PHP Warning] A non-numeric value encountered on line 1505 in file /gameserver/TM1/Killer/TMNF/xaseco/aseco.php

Any Ideas?

[Bueddl]

Hi Kripke

Code: Select all

End Round
....
[02/04,16:18:43] [RASP] Calculating ranks...
PHP Fatal error:  Uncaught ArgumentCountError: Too few arguments to function mysql_insert_id(), 0 passed in /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php on line 252 and exactly 1 expected in /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.compatibledatabase.php:104
Stack trace:
#0 /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php(252): mysql_insert_id()
#1 /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php(306): Rasp->getChallenges()
#2 /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php(618): Rasp->resetRanks()
#3 /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.rasp.php(633): Rasp->onEndrace(Object(Aseco), Array)
#4 /gameserver/TM1/Killer/TMNF/xaseco/aseco.php(1249): event_endrace(Object(Aseco), Array)
#5 /gameserver/TM1/Killer/TMNF/xaseco/aseco.php(1602): Aseco->releaseEvent('onEndRace', Array)
#6 /gameserver/TM1/Killer/TMNF/xaseco/aseco.php(1090): Aseco->endRace(Array)
#7 /gameserver/TM1/Killer/TMNF/xaseco/aseco.php(760): Aseco->executeCallbacks()
#8 /gameserver/TM1/Killer/TMNF/xase in /gameserver/TM1/Killer/TMNF/xaseco/plugins/plugin.compatibledatabase.php on line 104

(same error with the modernizer on other servers.

Easy one. The modernizer plugin messed up the wrap to mysql_insert_id here. Look for lines 100-102 in modernizer.inc.php and change them accordingly:

Code: Select all

// ...
if (!function_exists('mysql_insert_id')) {
	function mysql_insert_id($res){
		global $_db;
		return mysqli_insert_id($_db->conn);
	}
}
// ...

to

Code: Select all

// ...
if (!function_exists('mysql_insert_id')) {
	function mysql_insert_id($link = null){
		global $_db;
		return mysqli_insert_id(is_null($link) ? $_db->conn : $link);
	}
}
// ...

Code: Select all

[PHP Warning] A non-numeric value encountered on line 1490 in file /gameserver/TM1/Killer/TMNF/xaseco/aseco.php
[PHP Warning] A non-numeric value encountered on line 1505 in file /gameserver/TM1/Killer/TMNF/xaseco/aseco.php

The origin of this warning is in aseco.php inside the loadSettings function starting from line 218 in the upstream 1.16 version.
Values that are intended to be of numeric type should be read as intval(), but they aren't. You can simple go like this:

You warning are emmited by lines 1490 and 1505 of aseco.php. Look there for a read of settings var.
For lines 1490 and 1505 this both times is: $this->settings['show_recv_before'] .

Go to the corresponding line of loadSettings, which is 250 in upstream and change it accordingly:

Code: Select all

$this->settings['show_recs_before'] = $aseco['SHOW_RECS_BEFORE'][0];

to

Code: Select all

$this->settings['show_recs_before'] = intval($aseco['SHOW_RECS_BEFORE'][0]);

Normally php does a soft conversion from the string value it reads to the intvalue it is visiting at the lookup lines. This is, however, also done in the new php versions, but emitting a warning. You may encounter this in several more places (just have a look at your first log snippet, lines 1545 and so on are dealing with the counter part: show_recv_after).

Cheers,
Bueddl

[Kripke]

****, it seems to work. Thx a lot master Bueddl!!

[undef.de]

For me (UASECO) your code in xmlparser.inc.php at the function closeTag() wasn't working without "[PHP Notice]" like:

Code: Select all

[PHP Notice] Undefined index: SOMETHING on line NUM in file xmlparser.class.php

I had to replace the second is_array() with isset() to remove this "[PHP Notices]":

Code: Select all

	$top = $this->struct[$from];
	if (!is_array($this->struct[$to])) {
		$this->struct[$to] = array();
	}
	if (!isset($this->struct[$to][$from])) {
		$this->struct[$to][$from] = array();
	}
	$this->struct[$to][$from][] = $top;

Ubuntu and PHP/7.1.1-1+deb.sury.org~xenial+1

[Bueddl]

For me (UASECO) your code in xmlparser.inc.php at the function closeTag() wasn't working without "[PHP Notice]" like:

Code: Select all

[PHP Notice] Undefined index: SOMETHING on line NUM in file xmlparser.class.php

I had to replace the second is_array() with isset() to remove this "[PHP Notices]"
[...]
Ubuntu and PHP/7.1.1-1+deb.sury.org~xenial+1

Strange. Even with all warnings set to on I don't get this warning. My Setup is:
Gentoo, PHP/7.0.15, PHP/7.1.1, PHP/7.2.0-r1
Debian, PHP/7.2.0-r1

All of them compiled by myself. it should not matter anyway, but here is the build config:

Code: Select all

Build Date => Jan 25 2017 19:36:07                                                                                                                                                                                              
Configure Command =>  './configure'  '--prefix=/usr' '--build=x86_64-pc-linux-gnu' '--host=x86_64-pc-linux-gnu' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--datadir=/usr/share' '--sysconfdir=/etc' '--localstatedir=/var/lib' '--docdir=/usr/share/doc/php-7.1.1' '--htmldir=/usr/share/doc/php-7.1.1/html' '--prefix=/usr/lib64/php7.1' '--mandir=/usr/lib64/php7.1/man' '--infodir=/usr/lib64/php7.1/info' '--libdir=/usr/lib64/php7.1/lib' '--with-libdir=lib64' '--localstatedir=/var' '--without-pear' '--enable-maintainer-zts' '--with-fpm-acl' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--disable-gcov' '--enable-ctype' '--with-curl=/usr' '--enable-dom' '--without-enchant' '--enable-exif' '--enable-fileinfo' '--enable-filter' '--disable-ftp' '--with-gettext=/usr' '--without-gmp' '--enable-hash' '--without-mhash' '--with-iconv' '--enable-intl' '--enable-ipv6' '--enable-json' '--without-kerberos' '--enable-libxml' '--with-libxml-dir=/usr' '--enable-mbstring' '--with-mcrypt=/usr' '--with-onig=/usr' '--with-openssl=/usr' '--with-openssl-dir=/usr' '--disable-pcntl' '--enable-phar' '--enable-pdo' '--enable-opcache' '--without-pgsql' '--enable-posix' '--without-pspell' '--without-recode' '--enable-simplexml' '--disable-shmop' '--without-snmp' '--disable-soap' '--enable-sockets' '--with-sqlite3=/usr' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-fpm-systemd' '--without-tidy' '--enable-tokenizer' '--disable-wddx' '--enable-xml' '--enable-xmlreader' '--enable-xmlwriter' '--without-xmlrpc' '--without-xsl' '--enable-zip' '--with-zlib=/usr' '--disable-debug' '--enable-dba' '--without-cdb' '--without-db4' '--disable-flatfile' '--with-gdbm=/usr' '--disable-inifile' '--without-qdbm' '--with-freetype-dir=/usr' '--disable-gd-jis-conv' '--with-jpeg-dir=/usr' '--with-png-dir=/usr' '--without-xpm-dir' '--with-gd' '--with-imap=/usr' '--with-imap-ssl=/usr' '--without-interbase' '--with-ldap=/usr' '--without-ldap-sasl' '--with-mysqli=mysqlnd' '--with-mysql-sock=/var/run/mysqld/mysqld.sock' '--without-unixODBC' '--without-iodbc' '--without-oci8' '--without-pdo-dblib' '--with-pdo-mysql=mysqlnd' '--without-pdo-pgsql' '--with-pdo-sqlite=/usr' '--without-pdo-firebird' '--without-pdo-odbc' '--without-pdo-oci' '--with-readline=/usr' '--without-libedit' '--without-mm' '--with-pic' '--with-pcre-regex=/usr' '--with-pcre-dir=/usr' '--with-config-file-path=/etc/php/cli-php7.1' '--with-config-file-scan-dir=/etc/php/cli-php7.1/ext-active' '--disable-embed' '--enable-cli' '--disable-cgi' '--disable-fpm' '--without-apxs2' '--disable-phpdbg' 'build_alias=x86_64-pc-linux-gnu' 'host_alias=x86_64-pc-linux-gnu' 'CPPFLAGS='                                 
Server API => Command Line Interface                                                                                                                                                                                            

And for 7.2 (very, very reduced compared to my 7.0/7.1 build):

Code: Select all

Build Date => Jan 31 2017 16:11:43
Configure Command =>  './configure'  '--prefix=/var/gameserver/php/local' '--enable-mbstring' '--with-curl' '--with-openssl' '--enable-zip' '--enable-intl' '--with-mysqli' '--with-pdo-mysql' '--enable-pdo=shared' '--with-pdo-mysql=shared' '--with-pdo-sqlite=shared'
Server API => Command Line Interface

Anyway, seems like something else did already create the entry $this->struct[$to][$from]. As we did create the $this->struct[$to] array and as we are appending array elements using [] (precisely: $this->struct[$to][$from][] = $top;) it seems that some other code part is fooling here around. We are always using it and initializing it as an array, so the check should not fail....
However, using isset without a further check to is_array is erroneous as the append operation might fail or even silently convert to an array. You should think of adding the check again. So my purpose is this:

Code: Select all

   $top = $this->struct[$from];
   if (!is_array($this->struct[$to])) {
      $this->struct[$to] = array();
   }
   if (!isset($this->struct[$to][$from]) || !is_array($this->struct[$to][$from])) {
      $this->struct[$to][$from] = array();
   }
   $this->struct[$to][$from][] = $top;

This is suboptimal as it might discard the value already written to $this->struct[$to][$from] if it is not of array type, but this shouldn't be the case anyway!

Cheers ,

Bueddl

[undef.de]

Added

Code: Select all

 || !is_array($this->struct[$to][$from])

to the code brings no errors yet, so i keep it.

[Kripke]

Ahoi,

I got a small problem again, but I could not reproduce it^^. Maybe it's Xaseco problem or a combination with these fixes

Code: Select all

02/06,22:33:02] player ponur used chat command "/pmlog "
[02/06,22:33:12] player ponur used chat command "/admin pmlog"
PHP Fatal error:  Uncaught Error: [] operator not supported for strings in /gameserver/TM1/Nino/TMNF/xaseco/plugins/chat.admin.php:4827
Stack trace:
#0 /gameserver/TM1/Nino/TMNF/xaseco/aseco.php(1963): chat_admin(Object(Aseco), Array)
#1 /gameserver/TM1/Nino/TMNF/xaseco/aseco.php(1022): Aseco->playerChat(Array)
#2 /gameserver/TM1/Nino/TMNF/xaseco/aseco.php(760): Aseco->executeCallbacks()
#3 /gameserver/TM1/Nino/TMNF/xaseco/aseco.php(2562): Aseco->run('config.xml')
#4 {main}
  thrown in /gameserver/TM1/Nino/TMNF/xaseco/plugins/chat.admin.php on line 4827

Fatal error: Uncaught Error: [] operator not supported for strings in /gameserver/TM1/Nino/TMNF/xaseco/plugins/chat.admin.php:4827
Stack trace:
#0 /gameserver/TM1/Nino/TMNF/xaseco/aseco.php(1963): chat_admin(Object(Aseco), Array)
#1 /gameserver/TM1/Nino/TMNF/xaseco/aseco.php(1022): Aseco->playerChat(Array)
#2 /gameserver/TM1/Nino/TMNF/xaseco/aseco.php(760): Aseco->executeCallbacks()
#3 /gameserver/TM1/Nino/TMNF/xaseco/aseco.php(2562): Aseco->run('config.xml')
#4 {main}
  thrown in /gameserver/TM1/Nino/TMNF/xaseco/plugins/chat.admin.php on line 4827

[Bueddl]

Hey again

seems you got more than 14 lines of admin pm history. There is an (btw php5.6/7.x unreleated...) error in chat.admin.php.

Go to plugins/chat.admin.php, line 4832 and change:

Code: Select all

$msg = '';

to:

Code: Select all

$msg = array();

This should fix the problem

Background is, that $msg is an array of the lines to be outputted. The lines were your error occur seem to split the admin pm history in chunks of at max 14 lines. It is first initialized to this on line 4818. On line 4827 (the one present in your logs) a new line to this "buffer" is appended:

Code: Select all

$msg[] = ...

This is absolutely fine, but when there are more than 14 lines, you ran into the check on line 4829, which excecutes a rollback of the buffer:

Code: Select all

if (++$lines > 14) {
  $admin->msgs[] = $msg;
  $lines = 0;
  $msg = ''; // <-- your error is here ;-)
}

Whilst doing this, the $msg buffer is incorrectly set to an empty string, instead of an empty array.
Therefore, in the invocation of the append operation on line 4827, you got an attempt to call operator[] on a string, which is indeed ill-formed:

Code: Select all

[] operator not supported for strings 

However, I did not test this.

Cheers,
Bueddl

[Xymph]

This should fix the problem

Good catch.

[undef.de]

Go to plugins/chat.admin.php, line 4832 and change:

Code: Select all

$msg = '';

to:

Code: Select all

$msg = array();

This should fix the problem

Changed in UASECO as well, thanks Bueddl.

[NoTimeToDrive]

Hey.

First, a big thanks for keeping up to date the files!

But I would ask something for the certainty,
before I adjust things in my usbTMFserver.

What is the best thing I can do now?
Because I see a lot lines which are corrected.

Do I only need to replace the three files in post 1?
[2] patched files-including the memory leak fix - https://ftp.bueddl.de/tm/php7_patches/
Filename is XAseco-1:16

Or should I change more things?
Which I seen corrected here?

Thanks in advance

Greetings.