A few weeks ago, a started a new programming project has hobby: Picviz, a project to ease the creation, manipulation and visualization of parallel coordinates plot.

In short, parallel coordinates plot is a technique to visualize several dimensions in a 2D plan. When an event occurs, it has a set of variables and when it comes to deal with thousands if not million of data, it is a big challenge to understand them quickly and dig for correlation among variables and events. So here comes a 123 years old technique into the computer security field. I recommend you reading the Wikipedia page on the subject for more about this.

A few months ago, while doing research on IDS correlation with Pierre (Paper, Slides), we had fun writing PIG, The Prelude IDMEF Grapher on top of Prelude Python Easy bindings.

The application was very hackish and we could hardly automate the graph creation outside of writting stuff in Python, QT and using the IDMEF objects from Prelude.

Which is what I decided to fix.

I wanted an application that could be easy to automate the creation of graphs, able to deal with a large set of data, able to let people write easily their rendering and output plugins, a graph language as close as the graphviz dot language, python bindings to ease the creation of prototypes and use them for the graphical frontend.

I am pretty much close to the requirements, and you can download a (buggy? if so please report!) version with subversion:

svn co https://picviz.svn.sourceforge.net/svnroot/picviz/trunk picviz

To compile, you will need cmake, bison, flex, python2.5-dev (if you want python bindings), and QT with its python bindings (I took QT because I already knew gtk+ very well and wanted to give a try and especially not struggle with a canvas: please gtk+ people, I urge you to take goocanvas or anything you think better into the gtk+ sources!! I stopped writing gscore because of a lack of a decent canvas and I believe a lot of people are switching to QT because of this).

Then run:

# make install

And if you want python bindings:

# cd src/libpicviz/bindings/python/ && sudo ./setup.py install

Once you have done that, you can go into /samples and try the few examples there.

The language, as I said before is as close as graphviz as possible, you define your data and add properties in brackets after:

foobar [label="My label"]

It is devided in four sections:

  • header (optional): to set graph title, background etc..
  • engine (optional): to change the engine internals (careful!)
  • axes: to set your axes names, types and properties
  • data: where your data are
For example, a simple line would be:
header {
        title = "Simple graph";
}
axes {
        integer ax1 [label="First"];
        ipv4    ax2 [label="IP Addr"];
        string  ax3 [label="Third"];
}
data {
        ax1="12345", ax2="127.0.0.1", ax3="Some text" [color="red"];
}

Then run:
pcv -Tsvg blog-ex.pcv > blog-pcvex.svg

To produce the following image:


Now, if you want to visualize your iptables logs, you can run the perl script called "syslog-iptables2picviz" from the tools/ directory like this:
sudo perl tools/syslog-iptables2picviz /var/log/iptables > iptables.pcv

Which in my case produces the following image:

Here is a picture of 8000 events, which takes 2/3 seconds to generate on my laptop (3189.49 bogomips). The Zero is on top, so the time where this logs were taken is about 10:40 am (I am cheating because I know the time ;) ).

For information, the generation of 600 000 events took less than 10 seconds.

Now, in order to dig into the graph, I wrote a frontend in Python+QT. So I needed python bindings, which I wrote in the src/libpicviz/bindings directory. And the frontend is available in src/frontend.

To play with the first sample, you can run:
./picviz-gui.py ../../samples/test1.pcv

And when moving the mouse on the line, you will see the information:


Now please, before the upcoming 0.1 release, I would need testings and feedback, so please do!

Happy graphing!