It has been a bit more than a year that I started working on visualization for data mining. I strongly believe, that we are just humans, and having a good visualization system for your security events is not just a sexy toy. It is a real advantage for extracting relevant technical information from events overflow.

When I started with brouette, my little Prelude IDS pool event checker toy, I already had the spirit, but I did not really know what it could be, how it would grow etc..


After a few months, to play with the Ruby Language, I wrote "Ruined", the Ruby Iptables NEtwork Displayer. Actually, it was something that I previously wrote in bash (fw2foo), which is a nice script if you want to know how to deal with data structures in bash, callbacks etc.

After what, to ease the development of tools such as Ruined, I started GvGlue, to output dot language data for graphviz after quite some fight with bindings such as pydot and others to generate stuff such as a cluster into an other.

I wanted to learn either Python or Ruby at that time, I did not know which one I could take, so I wrote GvGlue in Python. And, well, sure GvGlue had a very nice API, but to code was terribly not something you wanted to see... unless you want to twist your nerves:


            try:
                for runner in self.data:
                    if runner.find("%d [label=" % subgraphid) != -1:
                        i = self.data.index(runner)
                        mystr = runner[:len(runner)-1] + ",%s=\"%s\"" % (propertyname, value) + "];"
                        self.data.pop(i)
                        if not getstr:
                            self.data.insert(i, mystr)


But at this time, crazy folks like haypo already started to use it to write awesome applications.


Because of Ruined limitations, and because GvGlue code scared me away, I rewrote it under the sexy name "GvGen". And it's been a long time GvGen became more interesting than GvGlue: keeping the KISS spirit and do not reinvent the wheel. Just output dot language. Allowing people to use dot, neato or other graphviz stuff.

GvGen is now part of graphviz ressources.

Last May, I went to Rennes for SSTIC event, which is a computer security symposium. And I saw the presentation Phil made about rtgraph3d.

I got quite excited: this brings a very interesting dimension to the way we can interpret events.

He actually used a hacked version of rtgraph3d along with scapy, to produce ipv6world for his presentation on the IPv6 mistakes and their famous routing headers type 0.

So I wrote a little patch to import graphviz dot language data using the pydot parser (which is quite good here). And now you can simply import those data into rtgraph3d.

I have some other patch I must finalize to bring the graphviz features (colors etc..) into rtgraph3d.

Now let's come back to Prelude stuff and IDMEF: Pierre wrote easy Prelude bindings for both perl and python into the libprelude.

So I decided to take his work to graph using the parallel axes representation and I took the following IDMEF values to write a graph:
  • alert.source(0).node.address(0).address
  • alert.target(0).node.address(0).address
  • alert.assessment.impact.severity
  • alert.classification.text

and a timeline to see the attack progression over 24 hours.

And that allows me to produce graphs like this:

And at the same time, I started to write Svp, a Sniffer that Visualize stuff written in Python.

The goal it to show on a parallel axes representation useful information to recognize patterns, such as a port scan.