A few weeks ago, a started a new programming project has hobby: Picviz, a project to ease the creation, manipulation and visualization of parallel coordinates plot.
In short, parallel coordinates plot is a technique to visualize several dimensions in a 2D plan. When an event occurs, it has a set of variables and when it comes to deal with thousands if not million of data, it is a big challenge to understand them quickly and dig for correlation among variables and events. So here comes a 123 years old technique into the computer security field. I recommend you reading
the Wikipedia page on the subject for more about this.
A few months ago, while doing research on IDS correlation with
Pierre (
Paper,
Slides), we had fun writing
PIG, The Prelude IDMEF Grapher on top of Prelude Python Easy bindings.
The application was very hackish and we could hardly automate the graph creation outside of writting stuff in Python, QT and using the IDMEF objects from Prelude.
Which is what I decided to fix.
I wanted an application that could be easy to
automate the creation of graphs, able to deal with a
large set of data, able to let people write easily their
rendering and output plugins,
a graph language as close as the
graphviz dot language,
python bindings to ease the creation of prototypes and use them for the
graphical frontend.
I am pretty much close to the requirements, and you can download a (buggy? if so please report!) version with subversion:
svn co https://picviz.svn.sourceforge.net/svnroot/picviz/trunk picviz
To compile, you will need cmake, bison, flex, python2.5-dev (if you want python bindings), and QT with its python bindings (I took QT because I already knew gtk+ very well and wanted to give a try and especially not struggle with a canvas: please gtk+ people, I urge you to take goocanvas or anything you think better into the gtk+ sources!! I stopped writing gscore because of a lack of a decent canvas and I believe a lot of people are switching to QT because of this).
Then run:
# make install
And if you want python bindings:
# cd src/libpicviz/bindings/python/ && sudo ./setup.py install
Once you have done that, you can go into /samples and try the few examples there.
The language, as I said before is as close as graphviz as possible, you define your data and add properties in brackets after:
foobar [label="My label"]
It is devided in four sections:
- header (optional): to set graph title, background etc..
- engine (optional): to change the engine internals (careful!)
- axes: to set your axes names, types and properties
- data: where your data are
For example, a simple line would be:
header {
title = "Simple graph";
}
axes {
integer ax1 [label="First"];
ipv4 ax2 [label="IP Addr"];
string ax3 [label="Third"];
}
data {
ax1="12345", ax2="127.0.0.1", ax3="Some text" [color="red"];
}Then run:
pcv -Tsvg blog-ex.pcv > blog-pcvex.svgTo produce the following image:
Now, if you want to visualize your iptables logs, you can run the perl script called "syslog-iptables2picviz" from the tools/ directory like this:
sudo perl tools/syslog-iptables2picviz /var/log/iptables > iptables.pcvWhich in my case produces the following image:
Here is a picture of 8000 events, which takes 2/3 seconds to generate on my laptop (3189.49 bogomips). The Zero is on top, so the time where this logs were taken is about 10:40 am (I am cheating because I know the time
).
For information, the generation of 600 000 events took less than 10 seconds.
Now, in order to dig into the graph, I wrote a frontend in Python+QT. So I needed python bindings, which I wrote in the
src/libpicviz/bindings directory. And the frontend is available in
src/frontend.
To play with the first sample, you can run:
./picviz-gui.py ../../samples/test1.pcvAnd when moving the mouse on the line, you will see the information:
Now please, before the upcoming 0.1 release, I would need testings and feedback, so please do!
Happy graphing!
As you can see, there is a red line, showing that I've got SSH scanned by one IP address at different times, with (hopefully!) failed authentication, on the root login.
Picviz 
I take graphviz API as a name reference to every property I can add to a line. To have a line with red color and with a width of 3, simply type:
I will be in London in two days attending 
Monday