EN:Linux Optimization Guide

From FrâG^.WIKI
Jump to: navigation, search

(Deutsche Version)

Server.png
Server.png
This article is related to game servers.


Optimizing the Linux system for game servers can help improving performance. The Linux kernel itself is only one part which can be optimized, so this guide tries to cover also other aspects. Please always keep in mind that there is no "golden way" of optimizing the system. Many things severely depends on hardware, type of game servers, plugins, and - even if neglected by most people - your personal idea of a perfect game server. Therefore this guide tries to present alternatives from time to time, which the advanced user might try out. If you are new to this field you should follow the default suggestions first, before trying any of the alternatives.

Also the different steps explained are not always leading to an improvement on all environments. So do not follow this guide from top to bottom and do all it says. Some things might even make everything worse in your case. Try one step at a time to see if it improves something. Sometimes also combinations of changes might do the trick, so you will have to experiment a little.

This guide is a rewrite of the "famous" Kernel howto, to bring it up-to-date and ease the maintenance of the guide. You can still find the old (better tested, but out-dated) howto here: EN:Linux_Kernel_Optimization

A note to potential authors: If you find any mistakes or want to post any improvements, please go ahead. This is a wiki, so you do not need the permission of the original author to edit it. Nevertheless you should maintain the fundamental ideas: a) this guide shall not dictate some specific thing to do or not to do - please always present possible options and alternatives; b) wherever possible this guide should be independent of the distribution, so please avoid distribution specific commands or hits (there is virtually always a distribution independent solution). But don't be scared, the worst thing can happen is the removal of your edit :-)


Contents

Defining your goals

Before you start you should be aware of the goals you want to achieve. This guide is primarily designed to optimize for the maximum possible quality of a game server. This requires you to keep the number of servers low, i.e. do not run more than one or two servers per cpu/core. Nevertheless some hints will be given for those who want to run as many as servers/slots possible on their given hardware.

Also you need to get straight what makes a good server for you. Most certainly, as high as possible FPS are not the right criterion. If you run hlds servers, you will want to reach stable 1000fps - but do not be fooled: hlds will never display values larger than 1000fps on the stats command. Therefore, if you managed to have perfectly constant 1000fps, you probably have FPS a little larger than 1000fps in reality. This will not make the server better, on the contrary: it will run (slightly) too fast. If you sometimes have FPS a little lower than 1000, your (real) FPS are probably as stable as before, the server is running no longer too fast, and you still have a perfect "LAN feeling" or what so ever. You can use the host_speeds command to obtain the real FPS (but only if you run it directly at server console, not via RCON).

If you run (orangebox) srcds servers, even 1000fps are too high (and now valve introduced kind of a limit to 500fps). The engine runs best, if FPS are matching exactly the tickrate (66 by default and not changeable for CS:S). Do not use slightly higher FPS, than your server will suffer from the so-called aliasing effect. If in doubt, run slightly lower. If yor FPS are not perfectly stable, "much" higher FPS can help to avoid the aliasing effect. "Much" in this case means something around 300fps, not more! Higher FPS will only use additional resources and not help with the server quality. (On the other hand, high FPS do not really hurt, so don't cry if someone still runs with 1000fps. But it is completely unnecessary to do so!)

Choosing the right distribution

It is very controversially discussed which distribution works best for game servers. Most Linux distributions are based on either Debian (e.g. Ubuntu) or Red Hat (e.g. CentOS, Fedora). Although people have had success with all distributions, it seems Debian based distributions work better out-of-the-box and therefore need less effort. If you have not yet decided which distribution you want to use, use the most recent Ubuntu desktop version.

Also you have to decide whether to use 32 bit or 64 bit. If you want to make use of a large amount of RAM (the limit is typically 3 GB for 32 bit, as the kernel reserves some address space), you will need a 64 bit system. Please note that even tough 32 bit Linux can access more than 3 and 4 GB, if the kernel is properly configured, it is strongly recommended not to use this feature for running game servers, as accessing memory above the 3 GB border will take additional time. Typically, srcds-based servers run better on 64 bit in any case. Only for hlds you need to decide between 32 and 64 bits. It seems still to be common practise to use 32 bit for hlds-based servers, but the differences do get smaller, so you will probably get a very good result with 64 bit either.

If you want to know whether your installed system is 32 or 64 bit, run: 

uname -m

If it prints x86_64 your system is 64 bit. If it prints i686 (or i586 or even i486 for very old systems), it is 32 bit. This does not mean your CPU is incapable of running a 64 bit system. All modern CPUs suitable for high-performance game server hosting support 64 bit.

Preparing your system

Depending on your distribution you will need to install some tools first. On most distributions, installing the following packages/programs should be sufficient (the actual package names might be slightly different, depending on the distribution):

Building your own kernel

Although recent kernels already provide a good performance out-of-the-box, it is still recommendable to compile your own kernel if you want to reach the maximum possible performance. If you don't know, which kernel version you have, run: 

uname -r

It will print out something like 2.6.35-24-generic. If your kernel is 2.6.32 or newer (like in this example), you can actually try using your current kernel and still build your own kernel afterwards if the result is not sufficient. For older kernels, this step is strongly recommended in any case.

As compiling a kernel is quite complex, this step has been put into a separate article, the Linux Kernel Compilation Guide.

Setting your servers to run with realtime scheduling

For maximum performance the operation system needs to know which processes are time critical and which not. By default every process is not considered to be time critical. The easiest way is to create a small cron job and let it run every 5 minutes or so. Put the following e.g. into /usr/local/sbin/resched.sh: 

#!/bin/sh

PROCESS_NAMES="srcds_linux srcds_i686 srcds_i486 srcds_amd hlds_i686 hlds_i486 hlds_amd"
for name in $PROCESS_NAMES; do
  PIDS=`pidof $name`
  for p in $PIDS; do
    chrt -f -p 98 $p
  done
done

# This is for the RT patches only, it does nothing on other kernels
PIDS=`ps ax | grep sirq-hrtimer | grep -v grep | sed -e "s/^ *//" -e "s/ .*$//"`
for p in $PIDS; do
  chrt -f -p 99 $p
done
PIDS=`ps ax | grep sirq-timer | grep -v grep | sed -e "s/^ *//" -e "s/ .*$//"`
for p in $PIDS; do
 chrt -f -p 51 $p
done

make the file executable: 

chmod 755 /usr/local/sbin/resched.sh

and put it into /etc/crontab: 

# m h dom mon dow user  command
[...]
*/5 * * * *  root  /usr/local/sbin/resched.sh > /dev/null 2>&1

Do not forget to restart your cron demon, e.g. on Debian: 

/etc/init.d/cron restart

Note: Using chrt to start the servers directly with the right scheduling does not work for the orangebox, therefore this generic method is described here.

Binding processes to cpu

In some environments it can help to assign a server to a specific cpu (resp. core). This can be done with the taskset command, e.g. 

taskset -c 0 ./srcds_run ...

which will force the game server to run on the first cpu. Replace the 0 by e.g. 1 to run the server on the second cpu (the cpu is counted starting from 0).

Beware, though you force the server to run on a specific cpu, you do not hinder other processes from running on the same cpu. For this reason, this probably makes the server worse in most cases. If you do not bind the server to a cpu, the kernel chooses automatically a free cpu to run the server on. This can cause the server to jump between cpus, but usually this does not do any harm. Maybe on systems with cache not being shared between cpus (i.e. AMD cpus and physically separated cpus) or with a cpu-dependent clock-source (i.e. tsc, see below how to change it) this can help stabilizing fps.

So try it out if you like, but it's not something you should start with!

Eliminate small Variations with "idler"

On some installations small variations in FPS remain after following this wiki. In this case it can help to have a low-priority process running that takes 100% cpu. Create a file called idler.c containing the following lines: 

int main() {
  while(1);
}

Then compile this program with the following command: 

gcc idler.c -o idler

You should run the program at low priority to aviod slowing down your system: 

nice ./idler &

The ampersand (&) will run the idler in the background.

Running only one idler even on systems with 2 or more cores should be enough (also if you run multiple game servers).

The variations should go away as log as the idler is running.

On some systems, it might be necessary to run one idler on each core, i.e.: 

nice taskset -c 0 ./idler &
nice taskset -c 1 ./idler &
nice taskset -c 2 ./idler &
nice taskset -c 3 ./idler &

Note: This might or might not work. Try it out, if it improves the situation for you. If not, do not use the idler. It will then only take away cpu resources.

Changing the clock source

A modern PC has various clocks that it can use for the timing of game servers (and everything else). Usually by default the TSC clock source is used, which is inside the cpu. This clock might be very well, and it might be very bad. It strongly depends on your cpu. In any case it hase the big advantage of being fast, as it can be read out directly without accessing another piece of hardware.

If you are not happy with the stability of your FPS, you can try using different clock sources to get a better result. To list all available clocks of your server, run: 

cat /sys/devices/system/clocksource/clocksource0/available_clocksource

You can then pick on of the listed clock names, e.g. "hpet", and set it to be the system clock source with this command: 

echo hpet >  /sys/devices/system/clocksource/clocksource0/current_clocksource

I recommend starting with hpet, but you can also try all other clock sources that are listed.

Trouble Shooting / FAQ

Links

Retrieved from "http://wiki.fragaholics.de/index.php/EN:Linux_Optimization_Guide"
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox