Today I spent a good part of the day migrating my stuff to the Raspberry..
The SD card I have installed has a total size 4GB so unfortunately not all of my things could fit..!
Also I am very concerned about the memory wear problem so I am thinking of adding a USB hard drive and mounting all of the most active parts of the SD filesystem there ( /var /tmp /home/user/public_html etc.. )

The server that I plan to replace is this ( first time posing for the camera.. )

Its a cabling mess really with a case made out of the retail motherboard box..

The raspberry takes much less space and will draw much much less power making this site more eco-friendly .. My UPS will also be able to power it up for days in the event of a power failure ..

I also conducted some experiments with wordpress on the raspberry , it performs worse than my current machine but my upload bandwidth is such a big bottleneck that 300ms of delay are barely noticeable ( I tested it from the computers of some of my friends and sometimes it takes almost 20 seconds for it to completely load ) ..

There are tools that cache the wordpress web pages or imporve PHP performance but basically all of them try to work around quering the database and PHP interpreter and serving static files instead..

To add to the CPU limitations of the Pi , using an SD card also means delays when writing/and reading to disk ( not to mention the finite read/write cycles ) when performing automatic caching operations , so instead of this I decided to add a script ( basically an alias for a wget call with the needed parameters ) that clones the whole site without hassle..

If you want to try it and after cloning AmmarServer ( with “git clone git://github.com/AmmarkoV/AmmarServer ” ) you can use the bash script provided in the repository and called “MirrorAmmarServer.sh”
Lets say that your WordPress blog is hosted on http://ammar.gr .
STEP 1 ) Being on the parent directory of AmmarServer issue “./MirrorAmmarServer.sh http://ammar.gr”
wget will gradually start to mirror the site into static files and create a subdirectory called ammar.gr/ where the files will be stored
STEP 2 ) Edit the file src/main.c and change char webserver_root[MAX_FILE_PATH]=”public_html/”; to char webserver_root[MAX_FILE_PATH]=”ammar.gr/”;
If you want you can change the DEFAULT_BINDING_PORT to another port ( you may want to set it to port 80 because the default is 8080 )
STEP 3 ) issue “./make && ./run_ammarserver” and voila your site will be availiable at http://127.0.0.1:8080 ( or any other port you choose )

Here is a comparison of pinging the Raspberry Pi and pinging my Intel Based Server PC on a LAN

The ping times of the intel system have an average of 0.158 ms while the raspberry pi has an average time of 0.543 ms , almost 3-4 times worse..

Here are some benchmarks conducted with chromium that give an overview of the performance difference ( all of the tests are done in a LAN so bandwidth is not a limiting factor ) :

Left : First Uncached Hit , Apache Serving regular WordPress on my regular PC ( 1.33MB transferred , DOMContent loaded @ 494 ms , onload @ 1.35s )
Right : First Uncached Hit , AmmarServer serving cached static files on Raspberry Pi ( 1.39MB transferred , DOMContent loaded @ 159 ms , onload @ 968ms )

Left : Second Hit after clicking Home , Apache Serving regular WordPress on my regular PC ( 22.67KB transferred , DOMContent loaded @ 566 ms , onload @ 2.04s )
Right : Second Hit after clicking Home , AmmarServer serving cached static files on Raspberry Pi ( 61.54KB transferred , DOMContent loaded @ 140 ms , onload @ 515ms )

Left : Apache Serving cached static files on my Regular PC
Right : AmmarServer serving cached static files on Raspberry Pi

Some things to notice is that in the static files version of the site , the image links on the menu bar that come directly from github , and the youtube embeds are the most laggy parts of the page and this distorts loading times. The onload event would drop by almost 400ms if they were part of the site..

Performance also gets better by increasing the priority of the webserver ( IRL decreasing it to -15 ) , removing cross-site content and making some other small tweaks. I have managed to drop times to around 50-60ms and I hope to further improve those numbers..

Network latency of course is the biggest bottleneck when dropping times so low and in any case even if i managed to serve the whole site from 60ms ( transaction latency ) + 150ms ( network latency ) in say 20ms ( transaction latency ) + 150 ms ( network latency ) the perceived difference from the average client will be very small..

Thats it for now..

To profile my server’s behaviour I am using chromium and the very handy Developer Tools that come integrated in it , Especially the Network Timeline feature is great for debugging this sort of things.. RedBot is also a very useful tool! that I use for debugging HTTP handshakes..

Although my implementation isn’t ready yet I am getting some very weird results when I emmit 304 Not Changed responses illustrated in the screenshots bellow ..!

The screenshots are that big due to my dual 1080p screen setup

I can’t seem to grasp the reason why a 304 response has such a large “Waiting Time” , Is it my Response header that is missing something , is it the whole Keep Alive thing not being handled properly , or is the waiting time supposed to mean something else in the graph..
How is it possible when only 200 byte headers get served that the response time is 4 times worse than when the full files get served from memory ?

I haven’t still figured it out and it is driving me crazy , but I will get to the bottom of this ..

UPDATE
After some tweaking around and with the help of redbot it turns out that the 304 Not Modified header should retransmit all of the original fields transmitted with the first “200 OK” message.. This means all of the E-Tag , Date etc fields.. I was missing out some of them so including them fixed the behaviour..

Hope you find it useful..!

Having the opportunity to first-hand test the pi I immediately apt-got git and cloned my AmmarServer repo to test it with the board

The results look very promising since the pi with ammarserver beats my current mini-itx intel atom based setup which is running the heavier apache webserver..

In more detail subsequently querying a static version of the the page you see right now ( as described in a previous post ) gave the following times..

Raspberry Pi using AmmarServer :
00:00:02.64 <- first uncached try with actual sd read time ( also worth to note that a Class 10 sd card is used )
00:00:00.706 <- rasp cached , no file reading involved
00:00:00.787 <- rasp cached , no file reading involved
00:00:00.818 <- rasp cached , no file reading involved
00:00:00.833 <- rasp cached , no file reading involved
00:00:00.788 <- rasp cached , no file reading involved
00:00:00.810 <- rasp cached , no file reading involved

My current webserver using apache ( and dynamic content [ php/mysql/wordpress ] ) :
00:00:02.802 <- ammar.gr with apache
00:00:01.561 <- ammar.gr with apache (some images etc. cached probably)

My current webserver using AmmarServer ( and static saved pages ) :
00:00:01.72 <- ammar.gr first uncached try with actual hd read time
00:00:00.678 <- ammar.gr with ammarserver cached , no file reading involved
00:00:00.652 <- ammar.gr with ammarserver cached , no file reading involved

As one can see the best times of the pi 00:00:00.706 are pretty damn close to the 00:00:00.652 of my current machine when the content is already cached in memory so I may be changing my web hosting setup soon..!

I will probably keep my current webserver as a platform to make dynamic webpages using the really nice GUI of wordpress and then pass static versions of them to the pi which will act as the always on webserver..
An additional thing that I didn't test was performance using more processes like a mumble server, an ftp server or an actual php/apache setup but..

When I'll order my own pi I will try all of them and blog back with my results..!

I am using it with synthesia a very nice program to learn and play the piano that uses midi files as input and has an impressive feature list and actually makes it “fun” to play and reduces very much the overall difficulty curve of learning to play complicated pieces..
Its learning pack is kind of expensive ( 40$ ) so I am planning to buying it to support its programmer , sometime soon but not right now.. For now its free mode is more than enough..!

One of the down-sides of synthesia is that it is a windows targeted application but thanks to the great work by the wine guys it plays with no problems at all using wine ( 1.5.10 )..! There is an other program which is linux oriented called linthesia but for some strange reason it refuses to start at all on my machine and from what I gather it has very limited capabilities compared to synthesia..!

The keyboard required no configuration on my part since it was recognized instantly when plugged in the USB port ( I am running Lubuntu 12.04 ) , the only “hack” needed was starting timidity in a pulseaudio friendly mode , done by issuing : timidity -A100 -iA -B2,8 -Os -EFreverb=0 2>&1 & and then chosing the keyboard from the synthesia menu and the testing one by one the midi out ports until sound came out..!

Thats about it..!
I am now trying to learn the following piece , in one day after so many years , I am close to the middle xD :

Yann Tiersen: Comptine d’un autre été

One of the tests that I’ve made is converting this wordpress site into static files using a firefox extension called scrapbook that can save entire web sites many pages deep as files and then using the saved files , serving them with ammarserver !

The result is that AmmarServer serving the cached files ( reading them from the disk every time ) is actually serving them twice as fast as when apache + php + wordpress dynamically create most of the content and serve the images etc in the same manner.. I am wondering how much better will I be able to do by caching and not querying the filesystem / hard disk subsystem at all..!

Attached are two screenshots taken using the numion stopwatch tool

AmmarServer running on port 8080 ,serving cached files reading them from disk and scoring 0.838 seconds

Regular apache + php + wordpress running on the site you currently see and scoring 1.888 seconds almost the double time..

These tests were executed on the same machine in a LAN network so no serious network unpredictable latencies exist and we are almost purely CPU bound..

Testing with caching in memory enabled for my site today gives the following times..
Apache + PHP + MySQL + WordPress : 2.323 secs
AmmarServer , cache enabled , first hit serving everything as a file : 1.383
AmmarServer , cache enabled , subsequent hits serving everything from memory : 0.762

using :
time wget http://ammar.gr/
and
time wget http://ammar.gr:8080/
just for the index.php file of wordpress shows the difference , 0.2s vs 0.004s

My /etc/X11/xorg.conf Screen section follows..!

Section “Screen”
Identifier “Screen0″
Device “Device0″
Monitor “Monitor0″
DefaultDepth 24
Option “NoLogo” “True”
Option “Coolbits” “1″
Option “RegistryDwords” “PowerMizerEnable=0×1; PerfLevelSrc=0×2222; PowerMizerLevel=0×3; PowerMizerDefault=0×3; PowerMizerDefaultAC=0×3″
SubSection “Display”
Depth 24
EndSubSection
EndSection

In other news since it is summer and I dont have many things to do..
I decided to rewrite from scratch AmmarServer ( http server ) for linux ..
The outdated windows version of AmmarServer can be found in my legacy site ..
In this new one I will be including keep-alive support , a better code layout , the ability to create virtual files via linking an arbitrary executable to the web server , and automatic caching web page queries to the memory.. This should be particularly handy to be used as a server in conjuction with the raspberry pi.. ( when I get my hands on one ) ..

The thing about virtual files is that I would like to make a virtual file called camera.jpg that when requested by the client would trigger an other frame grabbing application that would make a .jpg conversion of the image on the fly and serve it directly to the client without reading or writing to the flash memory/disk .. That way linking everything together all possible delays will be shoved off..!

AmmarServer github repository can be found here

Thats it for now..

While playing starcraft 2 with wine..
Kern.log ->

Jul 7 23:50:32 ammar-desktop kernel: [ 960.691374] INFO: task SC2.exe:2189 blocked for more than 120 seconds.
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691377] “echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message.
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691379] SC2.exe D ffffffff81806080 0 2189 1 0×00020000
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691382] ffff88018088bb78 0000000000000086 0000000000000001 ffff88018088bbc8
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691386] ffff88018088bfd8 ffff88018088bfd8 ffff88018088bfd8 0000000000013780
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691389] ffff8801b4c416f0 ffff88017ff92de0 0000000000000000 7fffffffffffffff
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691392] Call Trace:
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691398] [] schedule+0x3f/0×60
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691401] [] schedule_timeout+0x2a5/0×320
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691405] [] ? poll_freewait+0xe0/0xe0
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691407] [] ? __pollwait+0xf0/0xf0
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691410] [] wait_for_common+0xdf/0×180
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691414] [] ? try_to_wake_up+0×200/0×200
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691416] [] wait_for_completion+0x1d/0×20
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691494] [] os_acquire_semaphore+0×69/0×80 [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691550] [] _nv014565rm+0×9/0xe [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691596] [] ? _nv016163rm+0xce/0x1ee [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691638] [] ? _nv001029rm+0xbc0/0xca0 [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691680] [] ? _nv001063rm+0×66/0x2afb [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691722] [] ? _nv000937rm+0×26/0×147 [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691780] [] ? _nv001095rm+0x33d/0xa9f [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691835] [] ? rm_ioctl+0×76/0xf3 [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691888] [] ? nv_kern_ioctl+0x15c/0×490 [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691892] [] ? getnstimeofday+0×57/0xe0
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691945] [] ? nv_kern_compat_ioctl+0×21/0×30 [nvidia]
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691949] [] ? compat_sys_ioctl+0xad/0×240
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691953] [] ? compat_sys_time+0×25/0×60
Jul 7 23:50:32 ammar-desktop kernel: [ 960.691956] [] ? sysenter_dispatch+0×7/0x2e

Xorg log ->

[ 822.042] (WW) NVIDIA(0): WAIT (2, 6, 0×8000, 0x0000a4c8, 0x0000c408)
[ 828.683] [mi] EQ overflowing. Additional events will be discarded until existing events are processed.
[ 828.683]
Backtrace:
[ 828.704] 0: /usr/bin/X (xorg_backtrace+0×26) [0x7f73b42b3996]
[ 828.704] 1: /usr/bin/X (mieqEnqueue+0×263) [0x7f73b4294073]
[ 828.704] 2: /usr/bin/X (0x7f73b412b000+0x62ab4) [0x7f73b418dab4]
[ 828.704] 3: /usr/lib/xorg/modules/input/evdev_drv.so (0x7f73ac544000+0x5d88) [0x7f73ac549d88]
[ 828.704] 4: /usr/bin/X (0x7f73b412b000+0x8af87) [0x7f73b41b5f87]
[ 828.704] 5: /usr/bin/X (0x7f73b412b000+0xb0eca) [0x7f73b41dbeca]
[ 828.704] 6: /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f73b3451000+0xfcb0) [0x7f73b3460cb0]
[ 828.704] 7: (vdso) (0x7fff0bbcc000+0×707) [0x7fff0bbcc707]
[ 828.704] 8: (vdso) (0x7fff0bbcc000+0x7e5) [0x7fff0bbcc7e5]
[ 828.704] 9: (vdso) (__vdso_gettimeofday+0x2b) [0x7fff0bbcca1b]
[ 828.704] 10: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f73ad02c000+0x883a5) [0x7f73ad0b43a5]
[ 828.704] 11: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f73ad02c000+0xff713) [0x7f73ad12b713]
[ 828.704] 12: /usr/lib/xorg/modules/drivers/nvidia_drv.so (0x7f73ad02c000+0x4cf326) [0x7f73ad4fb326]
[ 828.704] 13: /usr/bin/X (0x7f73b412b000+0x11988c) [0x7f73b424488c]
[ 828.704] 14: /usr/bin/X (0x7f73b412b000+0x4a953) [0x7f73b4175953]
[ 828.704] 15: /usr/bin/X (0x7f73b412b000+0x4e8b1) [0x7f73b41798b1]
[ 828.705] 16: /usr/bin/X (0x7f73b412b000+0x3d6da) [0x7f73b41686da]
[ 828.705] 17: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0xed) [0x7f73b22e676d]
[ 828.705] 18: /usr/bin/X (0x7f73b412b000+0x3d9d1) [0x7f73b41689d1]
[ 828.705] [mi] These backtraces from mieqEnqueue may point to a culprit higher up the stack.
[ 828.705] [mi] mieq is *NOT* the cause. It is a victim.
[ 829.042] (WW) NVIDIA(0): WAIT (1, 6, 0×8000, 0x0000a4c8, 0x0000c408)
[ 829.042] [mi] Increasing EQ size to 512 to prevent dropped events.
[ 829.042] [mi] EQ processing has resumed after 43 dropped events.
[ 829.042] [mi] This may be caused my a misbehaving driver monopolizing the server’s resources.
[ 832.044] (WW) NVIDIA(0): WAIT (2, 6, 0×8000, 0x0000a4c8, 0x0000e348)
[ 839.044] (WW) NVIDIA(0): WAIT (1, 6, 0×8000, 0x0000a4c8, 0x0000e348)
[ 842.047] (WW) NVIDIA(0): WAIT (2, 6, 0×8000, 0x0000a4c8, 0x000002a8)
[ 849.047] (WW) NVIDIA(0): WAIT (1, 6, 0×8000, 0x0000a4c8, 0x000002a8)
[ 852.048] (WW) NVIDIA(0): WAIT (2, 6, 0×8000, 0x0000a4c8, 0x000062d8)
[ 859.048] (WW) NVIDIA(0): WAIT (1, 6, 0×8000, 0x0000a4c8, 0x000062d8)
[ 862.049] (WW) NVIDIA(0): WAIT (2, 6, 0×8000, 0x0000a4c8, 0×00008218)
[ 869.049] (WW) NVIDIA(0): WAIT (1, 6, 0×8000, 0x0000a4c8, 0×00008218)
[ 872.050] (WW) NVIDIA(0): WAIT (2, 6, 0×8000, 0x0000a4c8, 0x0000a158)
[ 879.050] (WW) NVIDIA(0): WAIT (1, 6, 0×8000, 0x0000a4c8, 0x0000a158)

I changed my xorg.conf Section “Device” to

Section “Device”
Identifier “Device0″
Option “Coolbits” “1″
Option “RegistryDwords” “PowerMizerEnable=0×1; PerfLevelSrc=0×2222; PowerMizerLevel=0×3; PowerMizerDefault=0×3; PowerMizerDefaultAC=0×3″
Driver “nvidia”
VendorName “NVIDIA Corporation”
EndSection

to set the graphics card powermizer setting to always use the lowest setting as a test but since it heats up a lot less , it is more quiet and the onboard electronics will probably last longer I am thinking of keeping it configured that way..!

NVidia forums remain under “maintenance”
nvnews.net has disabled new registrations..

just..

First of all setting Pulseaudio and OpenAL to a 48000Khz
using
sudo nano /etc/pulse/daemon.conf
and changing
; default-sample-rate = 44100
to -> default-sample-rate = 48000 ( removing the semicolon )

also changing the ;default-fragment-size-msec = 10
to -> default-fragment-size-msec = 5 ( once again removing the semicolon for all you noobs out there ;p [ just joking.. ] )

using
sudo nano /etc/pulse/daemon.conf
to change #frequency = 44100 to
frequency = 48000

issuing wine regedit to create the key
HKEY_CURRENT_USER/Software/Wine/Direct3D
and creating string keys with values as follow
KEY – VALUE
DirectDrawRenderer – opengl
Multisampling – disabled
OffScreenRenderingMode- pbuffer
PixelShaderMode – enabled
RenderTargetLockMode – disabled
StrictDrawOrdering – disabled
UseGLSL – disabled
VertexShaderMode – hardware
VertexShaders – hardware
VideoMemorySize – Your_video_memory_size_here_mine_is_895

the next step is setting cpu frequency scaling to performance mode before playing!
due to not having the gnome applets in lxde one can have the same functionality by apt-getting zenity and then running this cpufreq-govern-select.sh script ..

This with enabled VSYNC OpenGL setting in the NVIDIA driver and an emulated virtual desktop ( from winecfg ) makes Starcraft 2 run with no pulseaudio related sound glitches @ a solid 60fps on my PC , without these steps the game is almost unplayable with an average of 5-15fps..!

Fallout NewVegas on the other hand has very weird behaviour near areas with water , almost completely hanging , possibly due to shaders going haywire under wine/nvidia drivers/mesa/X ( you get the picture )
Luckily the game is very configurable and turning off all the water options solved the problems ..
To do so go to your /home/name/My Games/FalloutNV folder , make a backup copy of FALLOUT.ini and FalloutPrefs.ini and then go on and overwrite to make both of there water setting blocks look like this

[Water]
bUseWaterShader=0
bUseWaterReflections=0
bUseWaterRefractions=0
bUseWaterDepth=0
bUseWaterHiRes=0
bUseWaterDisplacements=0
bUseWaterLOD=0
bReflectExplosions=0
bAutoWaterSilhouetteReflections=0
bForceHighDetailReflections=0
bForceLowDetailReflections=0
fTileTextureDivisor=4.7500
fSurfaceTileSize=2048.0000
fNearWaterOutdoorTolerance=1024.0000
fNearWaterIndoorTolerance=512.0000
fNearWaterUnderwaterVolume=0.9000
fNearWaterUnderwaterFreq=0.3000
uNearWaterPoints=8
uNearWaterRadius=1000
uSurfaceFPS=12
iWaterReflectWidth=512
iWaterReflectHeight=512
bUseWaterReflectionBlur=0
iWaterBlurAmount=4

At last I can complete the underwater Volare mission ;P

Finally one last problem that I have faced using the latest NVIDIA drivers is my card dropping from the PCI bus , It is not only me who has the problem and thats good because it means its not hardware related and I have tried various things , turning powermizer on and of booting with kernel parameters “intel_iommu=off noirqbalance PCI=biosirq” , disabling CPU Virtualization features from BIOS , upgrading my bios to the latest version , enabling NVIDIA persistance mode , working with or without a second monitor physically connected , disabling irqbalance , disabling prelink , trying all the drivers ( NVIDIA-Linux-x86_64-285.05.09.run was the lowest driver version that seems to run with new kernels ) and many many many other things to find what causes this behaviour but it is probably something inside the NVIDIA binary blob so for now , just have to deal with it using patience..!
It is also not a heat problem since temperature is around 60degrees Celcious which is normal ..

Jul 7 16:42:42 ammar-desktop kernel: [ 3934.728909] show_signal_msg: 24 callbacks suppressed
Jul 7 16:42:42 ammar-desktop kernel: [ 3934.728913] lxpanel[1822]: segfault at 79 ip 000000000041fc92 sp 00007fffb7f17c30 error 4 in lxpanel[400000+2e000]
Jul 7 16:46:31 ammar-desktop kernel: [ 4163.707866] NVRM: GPU at 0000:05:00.0 has fallen off the bus.
Jul 7 16:46:31 ammar-desktop kernel: [ 4163.707873] NVRM: GPU at 0000:05:00.0 has fallen off the bus.
Jul 7 16:46:31 ammar-desktop kernel: [ 4163.707877] NVRM: GPU at 0000:04:00.0 has fallen off the bus.
Jul 7 16:46:31 ammar-desktop kernel: [ 4163.707880] NVRM: GPU at 0000:04:00.0 has fallen off the bus.

Some mentions of the bug :
https://bugzilla.kernel.org/show_bug.cgi?id=43050https://bugs.launchpad.net/ubuntu/+source/nvidia-graphics-drivers/+bug/882710?comments=all
http://www.nvnews.net/vbulletin/showthread.php?t=174759

https://wiki.archlinux.org/index.php/NVIDIA#.27.2Fdev.2Fnvidia0.27_Input.2FOutput_error

http://www.nvnews.net/vbulletin/showthread.php?t=167363