Category Archives: linux

Analyzing packets with help of tcpdump and ngrep

My recent assignment requires working with PHP script listening for connections from network-enabled devices using proprietary protocol.

When new device came in for testing noone knew why it’s data didn’t arrive into the system even though it used the same protocol as older devices. After confirming some details and checking every possible logs I’ve found I decided I need to go deeper – I’ve resorted to network monitoring.

tcpdump

I knew only few things about the device I needed to debug – the most important was it’s serial number which was hidden in each packet and information about it’s IP address and port (which I found out were changing quite often). My first thought was to dump packets originating from current IP address of the device:

tcpdump -i bond0 -vvv -X -s0 port 1025 and host xx.xx.xx.xx

Selected options stand for:
-i – interface to listen on
-vvv – maximum verbosity mode
-X – ASCII and hex content of a packet
-s0 – make sure to display full packet content (max 65535 bytes)
port 1025 and host xx.xx.xx.xx – filter only packets with specified src / dst addresses

The problem was I did not get any packets to work with – at the time I have set up actual IP address/port it has already changed. I could dump all packets arriving at PHP service port but with such a high rate of new packets I would need to dump hundreds of megabytes of data to get anything useful from it.

But this was about to change.

ngrep

Ngrep is an interesting tool which you can use almost as easily as grep but in network context. Instead of dumping whole lot of data to a file and then search through it looking for my pattern I was able to do this on the fly.

It was that easy. I could simply grep through all incoming packets to look for the device ID I was looking for. It didn’t matter if the IP address / port has changed, I could just run the tool and few minutes later analyze the results.

ngrep -X '0x00000932' -q -x

Options I have used:
-X – treat the pattern as hex value instead of regular expression
-q – display only packets I’m interested in
-x – display hex data (similarly to standard hex viewers)

After little fiddling I was able to retrieve few packets I could analyze, the results were similar to this packet:

T XX.XX.XX.XX:4186 -> XX.XX.XX.XX:12300 [AP]
 00 00 00 00 00 00 00 3e 08 01 00 00 01 49 51 d2 .......>.....IQ.
 97 5e 00 0d 1f 08 30 1d 9e 8a 20 01 32 00 97 0b .^....0... .2...
 00 0f 00 0a 05 01 01 45 01 f0 01 15 03 c8 00 03 .......E........
 b5 00 0d b6 00 08 42 35 3d 02 c7 00 00 00 09 f1 ......B5=.......
 00 00 65 93 00 01 00 00 e1 a8 ..e.......

Now, using protocol documentation I could go byte by byte and look for anything strange. Aha! There it was – one strange byte value pointing at undocumented feature which caused packet to be treated as invalid. Now I was sure that it wasn’t an issue on the system side (any protocol changes must first be applied on the system side) but on the device firmware side and after passing this information to technicians not so long later I have received updated protocol specification including this new byte value.

Gnome terminal embedded into desktop.

This is the last part of the Ubuntu desktop customizing, in this post I will cover embedding gnome terminal into your desktop and the simple autostart script for all of those features.

Before we start: Make sure you have CCSM (if you have not had one you can get it with sudo apt-get install compizconfig-settings-manager).

First important thing is to set up a new gnome terminal profile. To do so start terminal and select Edit->Profiles menu item. In the profile selection window select New and set a few key options:

(this screenshot can be your guide)

You must choose an unique name for your terminal window – I have chosen gnome_terminal and make sure to select option to keep the title unmodified. Second option to change is to remove the scroll bar completely – you will still be able to scroll the window with PgUp/PgDown keys.
After saving changes and selecting this profile for the current terminal leave it open and see, if everything is ok.

Now it’s time for some compiz fun!
Start CCSM and go to Window decoration setting and in the decoration_match input insert the following value:

(!title=desktop_terminal) 

Now go to Place windows and input the following settings as in the following screenshot:

The last rule to configure is in the Window rules tab. Similarily to the first option you ahve to fill it with the following value:

(title=desktop_terminal)

Note lack of ! sign

After all of this poking around close CCSM and take a look at your gnome terminal you set up at the beginning – from now on it should be sticking to the background with no window decoration at all!

Finally you have to correct the terminal position and start everything at system startup. For this task I wrote very short script:

#!/bin/bash
sleep 5
conky -c ~/conky/right
conky -c ~/conky/left
gnome-terminal --window-with-profile=desktop_terminal --geometry=110x40+95+20

Probably you will have to correct geometry parameter few times (took me quite a long time) before it will work for you. Save the script in your home directory and run at startup!

If you did everything as I wrote for you, you will get a nice desktop in no time. Now include own window selector (my preference is cairo dock) and you’re done.

Last note about why I have chosen Compiz and not Devil’s Pie for this task – the Devil’s Pie have some problems with disappearing terminal when you hit “show desktop” button, after trial end error I have forgot about it and made everything with compiz in much less time.
If you have any problem or want to point me out that I have forgotten about anything – let me know in the comments!

Conky setup with custom gauges, twitter and wlan status.

This is a follow-up of previous posts about conky and a summary.
Full source of the conky setup can be downloaded from here.

The full twitter script can be found in the previous post and along other important files in the archive.

The directory structure I am using is the following:

~/conky
     |- left
     |- right
     |- /scripts
         |- twit.py
         |- conky_lunatico.lua

It’s a simple structure using two conky instances for both sides of the screen with separate configuration files.
The most important bits of the left part are the following:

...
maximum_width 270
own_window yes
own_window_type override
own_window_transparent yes
own_window_hints undecorate,sticky,skip_taskbar,skip_pager,below
border_inner_margin 0
border_outer_margin 0
alignment top_left
...
 
//to set up the window properly and
...
${color1}${execi 60 python ~/conky/scripts/twit.py | fold -w45}
...

to include the twitter script.

Configuration of the right side has the following key items:

...
minimum_size 180 500
maximum_width 180
own_window yes
own_window_type override
own_window_transparent yes
own_window_hints undecorate,sticky,skip_taskbar,skip_pager,below
border_inner_margin 0
border_outer_margin 0
alignment tr
...
 
//as previously - setting up the window on the top-right side of the screen and
 
...
lua_load ~/conky/scripts/conky_lunatico.lua
lua_draw_hook_post main
...

To display the nice looking graphs.

And that’s pretty much of it!
For the graph position you will have to edit conky_lunatico.lua script to change values or change the data you are trying to display (I’m showing 7 separate graphs for each value, probably you will want to remove second WLAN graph from the script and config or change the name of your wireless adapter, it’s up to you.

To see it in action simply run conky -c ~/conky/right && conky -c ~/conky/left. For the how-to run this no your system startup and the last post of the overall desktop setup please take a look at the next post explaining Compiz and terminal setup.

Compiling conky with xmms2, lua and basic features support.

This is a second post about configuring my conky setup, yesterday I have posted a script displaying latest tweets from your Twitter account, today I will continue this topic today.

This machine is running Ubuntu 11.10 with Gnome classic desktop, compiz enabled and cairo dock installed.

In next posts I will describe how to create a setup like this:

For this setup I decided I will need the following features:

  • xmms2
  • wlan
  • lua with cairo

These features will be useful in later stages:

  • curl
  • RSS
  • weather

In my case I had few packages missing which were cairo-dock-dev, libiw-dev, libiw-dev and libtolua++5.1-dev.

After making sure you have all the libraries to compile conky you can download conky source, untar it into a temporary directory and run the following command:

./configure --prefix=/opt/conky --enable-xmms2 --disable-mpd --enable-nvidia --enable-weather-metar --enable-curl --enable-rss --enable-wlan --enable-lua --enable-lua-cairo --enable-lua-imlib2

A list of all features you can enable during compilation you can find issuing ./configure –help.

In case of any errors you should download missing package which name you will find in the error message.

When everything goes right you will see the following output:

 * X11:
  X11 support:      yes
  XDamage support:  yes
  XDBE support:     yes
  Xft support:      yes
  ARGB support      yes
 
 * Music detection:
  Audacious:        no
  BMPx:             no
  MPD:              no
  MOC:              yes
  XMMS2:            yes
 
 * General:
  math:             yes
  hddtemp:          yes
  portmon:          yes
  RSS:              yes
  Curl:             yes
  Weather
    METAR:          yes
    XOAP:           no
  wireless:         yes
  IBM:              no
  nvidia:           yes
  eve-online:       no
  config-output:    yes
  Imlib2:           yes
  ALSA mixer:       no
  apcupsd:          yes
  I/O stats:        yes
  ncurses:          yes
 
 * Lua (yes) bindings:
  Cairo:            yes
  Imlib2:           yes

This means you are almost done. Now do sudo make && sudo make install to install conky and try it typing conky in terminal.

In the next posts I will focus on conky scripts I used and how to embed gnome terminal in your desktop, stay tuned!

Drop huge amount of emails under linux

Last days I’ve run into an issue with customers email server. It was running very slowly so I’ve had to take a look at it.
Problem was caused by cron jobs sending error emails – 2 errors every 10 minutes.. since ’09..
This caused over 138,000(!) unwanted emails. I didn’t want to spend months at webmail deleting ~30 emails at once so I’ve quickly jump to shell and run rm *. I was quite suprised (I’m a linux noob) that it caused an Argument list too long error. find command ended up with the exactly same error message.

I thought I would try the workaround – bash script.

[~/mail]# nano remove.all
#!/bin/bash
for file in ./cur/*
do
rm $file
echo removed $file
done
[~/mail]# chmod +x remove.all
[~/mail]# ./remove.all

After about 20 minutes script has ended and it worked as it supposed to. Hooray!

I know there should be much easier way like drop whole directory instead, point of this way is to try out bash loops.