URL Shortners , my take using AmmarServer

Posted: 19th April 2013 by Ammar Qammaz in Post
0

Hello World,
been quite a long time since my last post here.. :)

This blog post is about URL shortners and specifically my URL shortner (MyURL) based on the infrastructure of AmmarServer
For more information on AmmarServer you can read this

URL shortners are internet tools that take long and impossible to remember URL strings like http://www.youtube.com/watch?v=lR9mNuLIPBU and add an easier to remember (or a random) alias that makes it shorter..! :) like http://tinyurl.com/d69n5sr or http://myurl.ammar.gr/fadetoblack ..

This web-service of course is a pretty straightforward one since it is just a lookup table with aliases pointing to strings and thats about it ,on the server side.. Thats also the reason why it is one of the first “service applications” of AmmarServer..

The code itself is simple and at the time around 420LOC , though I would like to reduce them even more..

The whole server executable is around 90KB ( taking efficiency seriously ;P )
and apart from the server binary there is also a plain-text file called myurl.db which contains the aliases and real links. Literally these are the two things you need and you got yourself a URL shortner.
Of course the db file is read only one time on startup and it is written every time a new alias is added from a user.

Basically what we do is spawn an AmmarServer instance on port 8080 and use an apache proxy to redirect traffic going to myurl.ammar.gr to 127.0.0.1:8080 . This can be easily done by enabling mod_proxy ( sudo a2enable mod_proxy , sudo a2enable mod_proxy_http ) and by adding a virtual host like the next code segment

<VirtualHost *:80>
ServerName myurl.ammar.gr
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass / http://127.0.0.1:8080/
ProxyPassReverse / http://127.0.0.1:8080/
</VirtualHost>

( Note : I would really like to make a proxy based on AmmarServer and probably will do to be able to get rid of apache completely )

Of course as I do with most things I tested MyURL against other popular URL Shortners , ( namely tinyurl.com , bit.ly , goo.gl , tr.im and ow.ly ) and found out a good performance advantage ..

As demonstrated below :

Task Shortening ->

time wget http://www.youtube.com/watch?v=lR9mNuLIPBU
2013-04-19 17:05:03 (840 KB/s) – `watch?v=lR9mNuLIPBU.1′ saved [232418]
real 0m0.755s
user 0m0.000s
sys 0m0.008s

time wget http://myurl.ammar.gr/fadetoblack
2013-04-19 17:50:19 (12.7 MB/s) – `fadetoblack.7′ saved [127/127]
real 0m0.066s
user 0m0.000s
sys 0m0.000s

time wget http://tinyurl.com/d69n5sr
2013-04-19 16:15:08 (815 KB/s) – `d69n5sr’ saved [232370]
real 0m1.082s
user 0m0.000s
sys 0m0.008s

time wget http://bit.ly/plIyZu
2013-04-19 16:18:03 (716 KB/s) – `plIyZu’ saved [232383]
real 0m1.038s
user 0m0.000s
sys 0m0.008s

time wget http://goo.gl/XAQqU
2013-04-19 16:19:22 (742 KB/s) – `XAQqU’ saved [232387]
real 0m0.825s
user 0m0.004s
sys 0m0.008s

time wget http://tr.im/42d2q
2013-04-19 16:23:29 (825 KB/s) – `42d2q’ saved [232693]
real 0m1.218s
user 0m0.000s
sys 0m0.012s

time wget http://ow.ly/kdVRX
2013-04-19 16:36:15 (806 KB/s) – `kdVRX’ saved [232907]
real 0m1.209s
user 0m0.004s
sys 0m0.008s

Of course this is not an accurate benchmark since myURL currently only has around 30 aliases while the other services have possibly millions , and also I am testing this from Crete , Greece , and the myURL server is in Athens Greece , while most of the others are on the US ( which implies a lot of “jet”lag ) , and network congestion might vary from time to time..

Then I took a look on the filesizes of the responses , which to my amazement was ~300KB .. Turns out that due to the Redirection using 301 headers wget was actually going on and retrieving the full youtube page so i then went on subtracting this time from each of the responses which yielded

myURL 0.066s <- myURL performance
tinyurl 0.327s
bit.ly 0.283s
goo.gl 0.07s <- this is also awesome
tr.im 0.463
ow.ly 0.454s

So that being said , considering my non-special hardware server , my plain DSL connection , my DynDNS DNS , my D-Link ADSL modem doing NAT and serving the data , myURL performance is overall superfast :) !

One question is how the f*$# is google going “as” fast ( and while providing a better service , analytics , user based “tracking” ;P etc ) and another one is how much faster could I go :) !
When running a test through a local loopback connection I get

time wget http://127.0.0.1:8080/fadetoblack
2013-04-19 17:27:05 (5.55 MB/s) – `fadetoblack.1′ saved [127/127]
real 0m0.010s

which is the maximum throughput the server can go at ( it is 7 times less than google’s net time ) If I only knew a little bit about how fast goo.gl go from inside their LAN , or what their software architecture is like :)

You can find MyURL here -> http://myurl.ammar.gr/ , although it is in alpha stage and thus will probably not be always online ( as I am performing tests on it , recompiling ,restarting etc ) ..!
Also ammar.gr is not exactly a high-availiability site , mainly thanks to my ISP :P
Of course as mentioned above myurl.ammar.gr is the same as calling http://ammar.gr:8080 . Of course calling myurl.ammar.gr also adds an overhead for the proxy thing to take place but for now it will have todo , Also I might change ports in the future so myurl.ammar.gr will be the thing to remember..

Picture of the day ;P

Posted: 14th March 2013 by Ammar Qammaz in Post
0

The Feeling when ..

OpenNI and ROS

Posted: 13th March 2013 by Ammar Qammaz in Post
Tags: , , , , ,
0

I’ve been doing my first steps using ROS for the last week
I have mixed feelings about it both as an architecture and implementation , I guess I will post them extensively another time..

What I wanted to record somewhere for someone that might face the same problem is the very strange behaviour of OpenNI after I installed ROS and its OpenNI libraries.. ( sudo apt-get install ros-groovy-openni-camera ros-groovy-openni-launch )

I’ve been getting some really weird behaviour , namely an Error (” Bad Parameter sent to the device! ” ) after the
StartGenerating call on the image generator.. This happens everywher including standard binaries of the PrimeSense codebase like NiViewer , but as well as my own wrapper ( https://github.com/AmmarkoV/RGBDAcquisition )

After some extensive tinkering it turns out that after ROS gets installed it takes over the “OpenNI stuff” and in order to use it as usual you have to first initialize it the ros way.. The weird thing is that this does not impact getting Depth frames that work correctly..

If you want to use it you should first initiate :

roscore
roslaunch openni_launch openni.launch &

and test if it works using -> rosrun rqt_reconfigure rqt_reconfigure

The same library with no modifications works correctly after the steps mentioned

I have also found some similar problems that might be different manifestations of the same problem..
So I really hope this helps someone, it took me almost two days to find the solution ..! :P

http://answers.ros.org/question/54907/openni-error-opening-video-stream/

Introducing Advanced Image Search for Linux!

Posted: 27th February 2013 by Ammar Qammaz in Post
Tags: , , , , ,
0

Ok the title is an overkill , but it is actually advanced when compared to plain search categorization using filenames and it “feels” a lot like google image search , so I thought if google calls it that way , why shouldn’t I..!

The idea is a tool for console based content-aware image searching much like the experience when using google advanced image search.. All this from the comfort and privacy of your terminal screen..
Of course such a cli tool could be leveraged by a GUI application to provide additional searching services in a more “user-friendly” way , and with that beeing said I am planning to make use of it in my FlashySlideshows project ..!

At last my huge random 4chan image folder will become automatically arranged! :)
I litteraly don’t know what I will do with all the time this will free ;P

sample queries using the library/tool are :


Click to see lsimg in action

Gathering all the images that “look like” 1360273766703.jpg ( a meme from meme generator ) into the Downloads/MemeTest dir..!

mv `lsimg -likeExt ~/Downloads/1360273766703.jpg 30 60 ~/Downloads/` ~/Downloads/MemeTest/


Click to see lsimg in action

Querying all “red” images
by using
lsimg -color red ~/Downloads/

Querying all images big enough for being used as a wallpaper
by using
lsimg -minDims 1920 1080 ~/Downloads/

Of course you can combine filters together for more complex queries and there are more parameters coming up ( like face detection , containers , GPS EXIF info parsing etc )

The project is licensed under GPLv3 , it is only targeted for the Linux platform
and you can download it from here : https://github.com/AmmarkoV/AdvancedImageSearch

(or by issuing)

git clone https://github.com/AmmarkoV/AdvancedImageSearch.git
cd AdvancedImageSearch
./get-dependencies.sh
./install.sh

The project currently only works on jpeg images (since it is in early alpha stage) and there are no other dependecies except libjpeg for now
I will probably have to add openCV linkage for use of the composite openCV face detectors and misc other stuff I want to include..

That’s it for now..!
Hope you enjoy..!

A few thoughts on the PS4

Posted: 21st February 2013 by Ammar Qammaz in Post
Tags: , , ,
0

ps4
Actual Sony patented technology

Playstation 4 was announced today after a show that was broadcasted live over the internet..
Unfortunately I can’t seem to find a recording of it somewhere , but when I do I will append it here..

Although I grew up playing with PS1 played only a little with PS2 and owned a PS3 it is now apparent that I will not be getting the PS4 :) ..

The main things that I didn’t like about it are..

- The controller “share” button! ( I wonder who came up with that idea.. )
- The fact that you can’t “share” your games with your friends any more :P (literally share by lending them the disc)
It appears this will not be the case , or so I read from news sites on the internet
- The social media , free advertisment on your friends , on you , on everyone attitude
- The “it will be solved in the cloud” approach on backwards compatibility with older generation devices ( let alone games you already purchased )
- The fact that you can’t turn your own device off unless you pull the plug.
- The mandatory social media logins , with everything this implies..
- The secondary “custom-chip” that performs “background-processing” , linked to your social media :p
- The fact that it will download things that it will decide its user might want in the future :P ( I bet it could also torrent-like share them with other users all in the name of “better performance” and at the users bandwidth cost )
- The playstation eye having a proprietary auxilary connection to the console and not a USB one..
- There were no price reports or “hands on” view..
- The marketing glossary of the presentation..

What I do like is :
- The video encoding “co-processor” for remote-operation ( although the PSVita sucks and it is a Wii-U ripoff idea )
I believe “hardware accelerated graphics output forwarding” would be a great platform-agnostic addon to modern digital devices , along with a way to transmit control messages it could definately be useful. Of course ssh and X-Forwarding are great but if you want to stream 3D graphics and full-hd movies they are kind of problematic..
- The stereo approach on 3d viewing , it will be nice in a future with devices like oculus rift or google glasses , if PS4 eye is of the same quality as PS3 eye it will be a great device , properly calibrated and synchronized , I really hope it will be usable by non PS4 hardware..
- The developers will probably have an easier job optimizing their code for all their platforms ( since there is basically only one common platform architecture :P ) , although if the playstation SDK and the black box it comes with costs nearly as much as the PS3 one ( 7500Euro+ ) , I still don’t think PS4 games will become easier to develop

What I’d like to note is that everything from Mac’s to Consoles is a PC nowadays..
As I said being a PS3 owner , I will not buy a PS4. I would like to buy its camera system if there was a way to connect it with a normal pc :P

That being said I am really looking forward to the Microsoft answer ( X-Box 720 probably streaming to their Surface tablets or w/e :P ) and the Steambox or “Piston” which is the platform that hopefully will win the 8th generation console wars..

Older Entries