Analyzing packets with help of tcpdump and ngrep

My recent assignment requires working with PHP script listening for connections from network-enabled devices using proprietary protocol.

When new device came in for testing noone knew why it’s data didn’t arrive into the system even though it used the same protocol as older devices. After confirming some details and checking every possible logs I’ve found I decided I need to go deeper – I’ve resorted to network monitoring.

tcpdump

I knew only few things about the device I needed to debug – the most important was it’s serial number which was hidden in each packet and information about it’s IP address and port (which I found out were changing quite often). My first thought was to dump packets originating from current IP address of the device:

tcpdump -i bond0 -vvv -X -s0 port 1025 and host xx.xx.xx.xx

Selected options stand for:
-i – interface to listen on
-vvv – maximum verbosity mode
-X – ASCII and hex content of a packet
-s0 – make sure to display full packet content (max 65535 bytes)
port 1025 and host xx.xx.xx.xx – filter only packets with specified src / dst addresses

The problem was I did not get any packets to work with – at the time I have set up actual IP address/port it has already changed. I could dump all packets arriving at PHP service port but with such a high rate of new packets I would need to dump hundreds of megabytes of data to get anything useful from it.

But this was about to change.

ngrep

Ngrep is an interesting tool which you can use almost as easily as grep but in network context. Instead of dumping whole lot of data to a file and then search through it looking for my pattern I was able to do this on the fly.

It was that easy. I could simply grep through all incoming packets to look for the device ID I was looking for. It didn’t matter if the IP address / port has changed, I could just run the tool and few minutes later analyze the results.

ngrep -X '0x00000932' -q -x

Options I have used:
-X – treat the pattern as hex value instead of regular expression
-q – display only packets I’m interested in
-x – display hex data (similarly to standard hex viewers)

After little fiddling I was able to retrieve few packets I could analyze, the results were similar to this packet:

T XX.XX.XX.XX:4186 -> XX.XX.XX.XX:12300 [AP]
 00 00 00 00 00 00 00 3e 08 01 00 00 01 49 51 d2 .......>.....IQ.
 97 5e 00 0d 1f 08 30 1d 9e 8a 20 01 32 00 97 0b .^....0... .2...
 00 0f 00 0a 05 01 01 45 01 f0 01 15 03 c8 00 03 .......E........
 b5 00 0d b6 00 08 42 35 3d 02 c7 00 00 00 09 f1 ......B5=.......
 00 00 65 93 00 01 00 00 e1 a8 ..e.......

Now, using protocol documentation I could go byte by byte and look for anything strange. Aha! There it was – one strange byte value pointing at undocumented feature which caused packet to be treated as invalid. Now I was sure that it wasn’t an issue on the system side (any protocol changes must first be applied on the system side) but on the device firmware side and after passing this information to technicians not so long later I have received updated protocol specification including this new byte value.

Bezpieczeństwo aplikacji z wykorzystaniem tokena sprzętowego

Ostatnio głośno się mówi o włamaniach na konta gmail lub facebook. Większość osób nie dba o bezpieczeństwo swoich haseł, korzystają z tego samego hasła w różnych miejscach lub korzystają z haseł słownikowych. Aby utrudnić ewentualne włamanie coraz więcej portali wprowadza dwuskładnikowe uwierzytelnienie (np. w formie wiadomości sms lub jednorazowych tokenów).

Co to jest token?

Token to zwykle losowy ciąg cyfr, które generowane są automatycznie w pewnych odstępach czasu, a które wymagane są podczas logowania do aplikacji. Stosuje je coraz więcej serwisów począwszy od google (token w formie aplikacji mobilnej), przez partypoker (token sprzętowy) na paypal.com (token sprzętowy w formie karty kredytowej) kończąc.

Google i paypal.com są znane prawie każdemu, kto korzysta z internetu, skupię się więc na pl.partypoker.com i postaram się opisać proces rejestracji i ogólne wrażenia z korzystania z serwisu.

Rejestracja odbywa się poprzez aplikację – klienta. Na uwagę zasługuje tutaj wybór hasła, a raczej test siły nowego hasła.
Hasło powinno spełniać następujące warunki:

  • co najmniej 5 znaków
  • maksymalnie 20 znaków
  • jedna cyfra
  • jedna litera
  • brak znaków interpunkcyjnych
  • nie może zawierać adresu email i nazwy użytkownika

Niestety wprowadzono limit długości hasła i zablokowano możliwość skorzystania ze znaków specjalnych co znacznie wpływa na jakość hasła. Test przechodzą bardzo proste hasła w formie 123456q co w bezpiecznej aplikacji nie powinno mieć miejsca.

Całą sytuację poprawia możliwość zamówienia tokena (niestety dostępny dla kont VIP), który dodaje drugi składnik logowania.
Od momentu aktywacji tokena mamy dwie możliwości – logować się dodając na końcu hasła aktualny token lub standardowe logowanie plus dodatkowy formularz, w którym wpisujemy nasz token.

Bezpieczeństwo w miejscach, gdzie operujemy prawdziwymi pieniędzmi jest priorytetem, pl.partypoker.com z opcją tokena wydaje się optymalnym rozwiązaniem. Gdyby dodać do tego zmianę polityki dotyczącej jakości hasła można by brać z nich przykład.

Polish translation for Ambrosia wordpress theme.

Ambrosia theme is the one you are looking at right now. I’m really happy with it and just today there was an 1.3.4 update with a couple of additional languages so I wanted to contribute.
You can get this theme from here or by searching the wordpress repository.

I have created the polish translation for this theme. You can download the language files from here: Ambrosia theme polish language files.

Simply install the Ambrosia theme and upload pl_PL.mo and pl_PL.po files to wp-content/themes/ambrosia/lang directory on your server.

WordPress GeSHi plugin optimization and modification.

Download link at the bottom.

Yesterdays research has pointed me one important issue – the WP-syntax plugin was taking way too long to process the homepage. And it wasn’t only the WP-Syntax but also other syntax-highlighting plugins including WP-GeSHi-highlight and Better WordPress Syntax Highlighter.
From few tested I found that WP-GeSHi-Highlight has the best performance but still – it wasn’t so great.
I was thinking – is there any way to speed up this plugin (or at least minimize the “damage”)?

The plugin itself is very well commented and is very simple, this is the simplified flow of the script:

  1. Plugin init
  2. Loop through posts to display and parse full text looking for <pre lang=”> tag
  3. If found – mark the place with unique identifier
  4. Replace every unique identifier with the GeSHi output

What if one of the posts doesn’t contain <pre lang=”> tag? The plugin still has to parse whole text to look for it. This doesn’t sound good if you want to highlight code only in a fraction of your posts.

Before I did anything else I wanted to run a couple of tests:

WP-GeSHi-Highlight disabled, didn’t parse anything – only initialization.


WP-GeSHi-Highlight enabled, parsed 3 posts.


WP-GeSHi-Highlight enabled, parsed all 10 posts.

After done with the profiling I decided it is worth to modify the plugin.
The new feature works in a very simple way, go to your dashboard and under Settings->WP-GeSHi-Highlight you can set the tag with which posts will be parsed.

The last thing you need to do is to tag all the posts with the code snippets and the plugin will do all the rest for you.

Keep your load times low and your wordpress will love you forever!

You can download the modified plugin here.

WordPress security with PHPIDS – plugin benchmark.

PHPIDS logoPHPIDS (PHP-Intrusion Detection System) is a simple to use, well structured, fast and state-of-the-art security layer for your PHP based web application. The IDS neither strips, sanitizes nor filters any malicious input, it simply recognizes when an attacker tries to break your site and reacts in exactly the way you want it to.

Security of your blog is very important and to keep your blog safe you must be very careful when installing new plugins. Even when you take all precautions you can never be sure that the plugin you have just installed is secure.
If you are worried about security there is a hope – Mute Screamer plugin. It’s very simple to use. Just install it and set few simple options to suit your needs and you’re done. From now on PHPIDS system will make sure that any input from your visitors will be stripped of any malicious code like SQL Injection or XSS attack.

Sounds good but you may ask – will it slow down my blog?
Well, it’s time to try it!

This is my test setup:

  • WordPress 3.3.2
  • P3 (Plugin Performance Profiler) – to profile wordpress installation
  • Akismet
  • Google XML Sitemaps
  • Better WordPress reCAPTCHA
  • Piwik Analytics
  • WP-Syntax
  • WP Super Cache
  • Mute Screamer

There were 5 main scenarios:

  1. No plugins – all plugins disabled
  2. IDS only idle – only Mute Screamer enabled at normal operation
  3. IDS only attack – only Mute Screamer enabled with attack detected
  4. All plugins – no cache – all plugins except cache enabled
  5. All plugins – with cache – all plugins including cache enabled

Each scenario was run 20 times on different parts of the blog each time.

Detailed results can be found in the following table:
[table id=1 /]
Nothing spectacular, right?
The following graph represents loading time of each wordpress segment in each scenario:

PHPIDS (via Mute Screamer plugin) doesn’t introduce any major weight to your blog as opposed to what you would think.
This breakdown proves the point, Mute Screamer is only a fraction of code needed to display the blog, the core still takes about 70% of resources:

Let’s look at the memory usage and execution of the plugins:

This also proves that the additional security layer will not slow your blog more that any other ordinary plugin (please note that left axis is in logarithmic scale) and even under attack conditions the script doesn’t introduce any significant stress on the system (this can be very important if you want to hide the fact you are using IDS).

Before running those benchmarks I’ve been very skeptical about this plugin but now I’m pretty sure the profit from the increased security costs me only a fraction of what I thought it would.

(During the tests and writing this post the PHPIDS has blocked 6 XSS attacks from the automated bots from all around the world)

Gnome terminal embedded into desktop.

This is the last part of the Ubuntu desktop customizing, in this post I will cover embedding gnome terminal into your desktop and the simple autostart script for all of those features.

Before we start: Make sure you have CCSM (if you have not had one you can get it with sudo apt-get install compizconfig-settings-manager).

First important thing is to set up a new gnome terminal profile. To do so start terminal and select Edit->Profiles menu item. In the profile selection window select New and set a few key options:

(this screenshot can be your guide)

You must choose an unique name for your terminal window – I have chosen gnome_terminal and make sure to select option to keep the title unmodified. Second option to change is to remove the scroll bar completely – you will still be able to scroll the window with PgUp/PgDown keys.
After saving changes and selecting this profile for the current terminal leave it open and see, if everything is ok.

Now it’s time for some compiz fun!
Start CCSM and go to Window decoration setting and in the decoration_match input insert the following value:

(!title=desktop_terminal) 

Now go to Place windows and input the following settings as in the following screenshot:

The last rule to configure is in the Window rules tab. Similarily to the first option you ahve to fill it with the following value:

(title=desktop_terminal)

Note lack of ! sign

After all of this poking around close CCSM and take a look at your gnome terminal you set up at the beginning – from now on it should be sticking to the background with no window decoration at all!

Finally you have to correct the terminal position and start everything at system startup. For this task I wrote very short script:

#!/bin/bash
sleep 5
conky -c ~/conky/right
conky -c ~/conky/left
gnome-terminal --window-with-profile=desktop_terminal --geometry=110x40+95+20

Probably you will have to correct geometry parameter few times (took me quite a long time) before it will work for you. Save the script in your home directory and run at startup!

If you did everything as I wrote for you, you will get a nice desktop in no time. Now include own window selector (my preference is cairo dock) and you’re done.

Last note about why I have chosen Compiz and not Devil’s Pie for this task – the Devil’s Pie have some problems with disappearing terminal when you hit “show desktop” button, after trial end error I have forgot about it and made everything with compiz in much less time.
If you have any problem or want to point me out that I have forgotten about anything – let me know in the comments!

Conky setup with custom gauges, twitter and wlan status.

This is a follow-up of previous posts about conky and a summary.
Full source of the conky setup can be downloaded from here.

The full twitter script can be found in the previous post and along other important files in the archive.

The directory structure I am using is the following:

~/conky
     |- left
     |- right
     |- /scripts
         |- twit.py
         |- conky_lunatico.lua

It’s a simple structure using two conky instances for both sides of the screen with separate configuration files.
The most important bits of the left part are the following:

...
maximum_width 270
own_window yes
own_window_type override
own_window_transparent yes
own_window_hints undecorate,sticky,skip_taskbar,skip_pager,below
border_inner_margin 0
border_outer_margin 0
alignment top_left
...

//to set up the window properly and
...
${color1}${execi 60 python ~/conky/scripts/twit.py | fold -w45}
...

to include the twitter script.

Configuration of the right side has the following key items:

...
minimum_size 180 500
maximum_width 180
own_window yes
own_window_type override
own_window_transparent yes
own_window_hints undecorate,sticky,skip_taskbar,skip_pager,below
border_inner_margin 0
border_outer_margin 0
alignment tr
...

//as previously - setting up the window on the top-right side of the screen and

...
lua_load ~/conky/scripts/conky_lunatico.lua
lua_draw_hook_post main
...

To display the nice looking graphs.

And that’s pretty much of it!
For the graph position you will have to edit conky_lunatico.lua script to change values or change the data you are trying to display (I’m showing 7 separate graphs for each value, probably you will want to remove second WLAN graph from the script and config or change the name of your wireless adapter, it’s up to you.

To see it in action simply run conky -c ~/conky/right && conky -c ~/conky/left. For the how-to run this no your system startup and the last post of the overall desktop setup please take a look at the next post explaining Compiz and terminal setup.

Compiling conky with xmms2, lua and basic features support.

This is a second post about configuring my conky setup, yesterday I have posted a script displaying latest tweets from your Twitter account, today I will continue this topic today.

This machine is running Ubuntu 11.10 with Gnome classic desktop, compiz enabled and cairo dock installed.

In next posts I will describe how to create a setup like this:

For this setup I decided I will need the following features:

  • xmms2
  • wlan
  • lua with cairo

These features will be useful in later stages:

  • curl
  • RSS
  • weather

In my case I had few packages missing which were cairo-dock-dev, libiw-dev, libiw-dev and libtolua++5.1-dev.

After making sure you have all the libraries to compile conky you can download conky source, untar it into a temporary directory and run the following command:

./configure --prefix=/opt/conky --enable-xmms2 --disable-mpd --enable-nvidia --enable-weather-metar --enable-curl --enable-rss --enable-wlan --enable-lua --enable-lua-cairo --enable-lua-imlib2

A list of all features you can enable during compilation you can find issuing ./configure –help.

In case of any errors you should download missing package which name you will find in the error message.

When everything goes right you will see the following output:

 * X11:
  X11 support:      yes
  XDamage support:  yes
  XDBE support:     yes
  Xft support:      yes
  ARGB support      yes

 * Music detection:
  Audacious:        no
  BMPx:             no
  MPD:              no
  MOC:              yes
  XMMS2:            yes

 * General:
  math:             yes
  hddtemp:          yes
  portmon:          yes
  RSS:              yes
  Curl:             yes
  Weather
    METAR:          yes
    XOAP:           no
  wireless:         yes
  IBM:              no
  nvidia:           yes
  eve-online:       no
  config-output:    yes
  Imlib2:           yes
  ALSA mixer:       no
  apcupsd:          yes
  I/O stats:        yes
  ncurses:          yes

 * Lua (yes) bindings:
  Cairo:            yes
  Imlib2:           yes

This means you are almost done. Now do sudo make && sudo make install to install conky and try it typing conky in terminal.

In the next posts I will focus on conky scripts I used and how to embed gnome terminal in your desktop, stay tuned!

Latest tweets in your conky setup.

After getting a clean Ubuntu installation I decided to make few tweaks including a custom conky script which included a Twitter feed.

I made a quick research I found out the easiest solution using RSS feed is outdated so I have had to find the other way – and I did here.
As you can read there the only requirement is python-twitter package (and ofcourse python itself).

I have updated the code as it went obsolete when Twitter staff decided to complicate their API even more.

To make this script work you need a few of things:

1. Twitter account (duh)

2. Create Twitter application

3. Receive access tokens

After you will receive all required tokens you have to put them in marked place in the code below.

'''
Original author: Travis Moore (@travist120) from http://travist120.wordpress.com/
Updated by: Peter Kasperski (bolo130@o2.pl) from http://kasperski-web.pl

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program.  If not, see 

'''
import os
import string
import twitter

#set your API access details here.
consumer_key='CONSUMER KEY'
consumer_secret='CONSUMER SECRET'
access_token = 'ACCESS TOKEN'
access_token_secret = 'ACCESS TOKEN SECRET'

#how many tweets you want displayed at a time
twitterSize = 9;

#do not edit beyond this line.
api = twitter.Api(consumer_key, consumer_secret, access_token, access_token_secret)
status = api.GetFriendsTimeline()
i = 0
for s in status:
	i = i+1
	print s.user.name.encode("utf-8"), "(@"+s.user.screen_name.encode("utf-8")+"):"

	print s.text.encode("utf-8")
	print
	if i == twitterSize:
		break

The easy way to include it in your conky script is the following:

${execi 60 python ~/conky/scripts/twit.py | fold -w45}

If you will get any error try to run the script alone and make sure you have all required python libraries installed.

irssi encoding problem.

If you have a problem with encoding in irssi there is an easy way to fix it.
In my setup I am running irssi inside a screen session and connecting from windows7 computer via Putty.

First of all you have to make sure your Putty has correct encoding set, you can check it in configuration dialog under Window -> Translation page.

Second thing you need to do is to start screen with unicode support, you can do this with -U switch.

Third and the last thing you need to do is to set correct encoding in irssi itself, just do /set term_charset utf-8 in chat window.

From now on your irssi should have full unicode support.