Toady's blog

Picviz::Dshield

Sebastien Tricaud — Wed, 20 Aug 2008 19:51:00 +0000

I have written a Perl module to access to Dshield data. The idea behind that is to do some special reactions to stuff dshield sees for Picviz, such as coloring lines in red it they match.

To use the class you simply tell perl to use the class, create a new object like this:

use Picviz::Dshield;
$dshield = Picviz::Dshield->new();

This will actually download the list of IP and port trends from my daily updated wallinfire.net mirror (to avoid bothering dshield.org).

Then, to check wether an IP match a dshield data or not, you can call:


if ($dshield->ip_check("192.168.1.42")) {
        print "The IP address is in the Dshield database";
} else {
        print "The IP address is not in the Dshield database";
}

So without any knowledge of how dshield works etc, you can easily embed and script with perl a connection to it.

After that, I modified the ssh authentication script to see wether my server was attacked by IP addresses in Dshield, and I produced to following graph:
As you can see, there is a red line, showing that I've got SSH scanned by one IP address at different times, with (hopefully!) failed authentication, on the root login.

All this is available in the tools/ directory of the latest Picviz trunk. To install the Dshield perl module, simply type:


$ cd trunk/tools/Picviz-Dshield
$ perl Makefile.PL
$ make
$ sudo make install

and you can test your ssh connections using the ssh connection script in tools/auth/ :


$ perl ssh-authdshielded2pcv.pl /var/log/auth.log > auth.pcv
$ picviz-gui auth.pcv

Picviz news

Sebastien Tricaud — Mon, 18 Aug 2008 17:47:00 +0000

While I am in hollidays in Canada, I would like to share some Picviz development news:

Picviz entered Fedora CVS and will be available to Fedora users for next release. Package seems to have been backported until Fedora core 7. I would like to congratulate folks at Redhat who have been amazingly reactive and helped me to improve my cmake skills and make sure Picviz can be installed on every architecture.
While I was in the plane I added the penwidth property in trunk. So this will be in the September 0.3 release. As you can guess in the name, it increases the line width. This picture illustrates the property:

I take graphviz API as a name reference to every property I can add to a line. To have a line with red color and with a width of 3, simply type:

data1="foo",data2="bar",...,dataN="12345" [color="red",penwidth="3"];

Trying xkcd style

Sebastien Tricaud — Mon, 11 Aug 2008 21:19:00 +0000

I though it would be easier to draw this, damn! I am so bad

Picviz 0.2 is out!

Sebastien Tricaud — Thu, 07 Aug 2008 12:37:00 +0000

From release notes:

Picviz is a parallel coordinates plotter which enables easy scripting from various input (tcpdump, syslog, iptables logs, apache logs, etc..) to visualize your data and discover interesting results quickly.

Picviz helps you to create, automate and understand parallel coordinates plots. Its primary goal is to graph data in order to be able to quickly analyze problems and find correlations among variables. With security analysis in mind, the program has been designed to be very flexible, able to graph millions of events.

The language is designed to be close to the graphviz graph description language.

Picviz features a language to describe your graphs; An engine producing images in parallel coordinates from this language; Python bindings to gather calculated data, raw data and a frontend written in Python and QT4.

You can download the program from its project page.

Picviz trunk news

Sebastien Tricaud — Wed, 06 Aug 2008 22:42:00 +0000

I had a plplot plugin kept secretly that I decided to put in trunk.

This plugin allows you to generate several kind of images, such as PNG. When you are dealing with a huge set of data, this kind of plugin is recommended. Demonstration:

toady@marcadet:~/local/scm/svn/picviz/trunk/samples$ pcv -Tplplot test1.pcv 

Plotting Options:
 < 1> gcw        Gnome Canvas Widget
 < 2> ps         PostScript File (monochrome)
 < 3> psc        PostScript File (color)
 < 4> xfig       Fig file
 < 5> hp7470     HP 7470 Plotter File (HPGL Cartridge, Small Plotter)
 < 6> hp7580     HP 7580 Plotter File (Large Plotter)
 < 7> lj_hpgl    HP Laserjet III, HPGL emulation mode
 < 8> pbm        PDB (PPM) Driver
 < 9> null       Null device
 <10> mem        User-supplied memory device
 <11> svg        Scalable Vector Graphics (SVG 1.1)
 <12> xcairo     Cairo X Windows Driver
 <13> pdfcairo   Cairo PDF Driver
 <14> pscairo    Cairo PS Driver
 <15> svgcairo   Cairo SVG Driver
 <16> pngcairo   Cairo PNG Driver

Enter device number or keyword: 16
Enter graphics output file name: foobar.png
toady@marcadet:~/local/scm/svn/picviz/trunk/samples$

By the way, for people who missed it:

The project page is http://www.wallinfire.net/picviz
You can register to the mailing list at: http://www.wallinfire.net/cgi-bin/mailman/listinfo/picviz
There is an IRC channel on Freenode: #picviz

And the SVN repository moved away from sourceforge to wallinfire:

$ svn co http://www.wallinfire.net/svn-picviz/trunk picviz

Hum, and the picture of the day: a nmap scan

Best libprelude release ever released!

Sebastien Tricaud — Tue, 05 Aug 2008 21:58:00 +0000

Today libprelude 0.9.19 was release (read notes).

To me this is the best release ever because not only the library is very mature, but it allows you to use what we' ve called the easy bindings.

In short, with the easy bindings, you can install prelude, use its components and with a bit of Python/Lua/Ruby/Perl (swig allows other languages) to do:


import PreludeEasy
from PreludeEasy import ConnectionPool, Connection

c = PreludeEasy.ClientEasy("YourEventReader", PreludeEasy.Client.IDMEF_READ)

c.SetFlags(PreludeEasy.Client.CONNECT)


pool = c.GetConnectionPool()
pool.AddConnection(Connection("127.0.0.1"))


idmef = PreludeEasy.IDMEF()


while 1:

    ret = c.RecvIDMEF(idmef, 100)

    if ret and not idmef.Get("heartbeat.create_time"):

        print "Alert:%s" % (idmef.Get("alert.classification.text") or "(unknown)")

Picviz 0.1 released!

Sebastien Tricaud — Mon, 04 Aug 2008 14:38:00 +0000

Picviz 0.1 is now officially released!

You can grab the first version on the project page. And see stuff you can do with it with a page I am currently writing on the DNS Kaminsky attack.

Enjoy!

Behave like Apple, catch hackers interest!

Sebastien Tricaud — Sun, 03 Aug 2008 21:09:00 +0000

So I've learned tonight that the company that made Snow White sleep reiterates canceling talks during black hats.

Yes, they reiterate because Daniel's talk at Eusecwest 2007 was also canceled for an obscure reason.

Did someone told Apple that this is exactly the right behavior to catch hackers interest?

HD Moore owned ;)

Sebastien Tricaud — Wed, 30 Jul 2008 15:11:00 +0000

This news is quite funny!

One of BreakingPoint's servers was forwarding DNS (Domain Name System) traffic to the AT&T server, so when it was compromised, so was HD Moore's company

Picviz, or the art of parallel coordinates

Sebastien Tricaud — Sun, 20 Jul 2008 20:01:00 +0000

A few weeks ago, a started a new programming project has hobby: Picviz, a project to ease the creation, manipulation and visualization of parallel coordinates plot.

In short, parallel coordinates plot is a technique to visualize several dimensions in a 2D plan. When an event occurs, it has a set of variables and when it comes to deal with thousands if not million of data, it is a big challenge to understand them quickly and dig for correlation among variables and events. So here comes a 123 years old technique into the computer security field. I recommend you reading the Wikipedia page on the subject for more about this.

A few months ago, while doing research on IDS correlation with Pierre (Paper, Slides), we had fun writing PIG, The Prelude IDMEF Grapher on top of Prelude Python Easy bindings.

The application was very hackish and we could hardly automate the graph creation outside of writting stuff in Python, QT and using the IDMEF objects from Prelude.

Which is what I decided to fix.

I wanted an application that could be easy to automate the creation of graphs, able to deal with a large set of data, able to let people write easily their rendering and output plugins, a graph language as close as the graphviz dot language, python bindings to ease the creation of prototypes and use them for the graphical frontend.

I am pretty much close to the requirements, and you can download a (buggy? if so please report!) version with subversion:

svn co https://picviz.svn.sourceforge.net/svnroot/picviz/trunk picviz

To compile, you will need cmake, bison, flex, python2.5-dev (if you want python bindings), and QT with its python bindings (I took QT because I already knew gtk+ very well and wanted to give a try and especially not struggle with a canvas: please gtk+ people, I urge you to take goocanvas or anything you think better into the gtk+ sources!! I stopped writing gscore because of a lack of a decent canvas and I believe a lot of people are switching to QT because of this).

Then run:

# make install

And if you want python bindings:

# cd src/libpicviz/bindings/python/ && sudo ./setup.py install

Once you have done that, you can go into /samples and try the few examples there.

The language, as I said before is as close as graphviz as possible, you define your data and add properties in brackets after:

foobar [label="My label"]

It is devided in four sections:

header (optional): to set graph title, background etc..
engine (optional): to change the engine internals (careful!)
axes: to set your axes names, types and properties
data: where your data are

For example, a simple line would be:

header {
        title = "Simple graph";
}
axes {
        integer ax1 [label="First"];
        ipv4    ax2 [label="IP Addr"];
        string  ax3 [label="Third"];
}
data {
        ax1="12345", ax2="127.0.0.1", ax3="Some text" [color="red"];
}

Then run:
pcv -Tsvg blog-ex.pcv > blog-pcvex.svg

To produce the following image:

Now, if you want to visualize your iptables logs, you can run the perl script called "syslog-iptables2picviz" from the tools/ directory like this:
sudo perl tools/syslog-iptables2picviz /var/log/iptables > iptables.pcv

Which in my case produces the following image:

Here is a picture of 8000 events, which takes 2/3 seconds to generate on my laptop (3189.49 bogomips). The Zero is on top, so the time where this logs were taken is about 10:40 am (I am cheating because I know the time ).

For information, the generation of 600 000 events took less than 10 seconds.

Now, in order to dig into the graph, I wrote a frontend in Python+QT. So I needed python bindings, which I wrote in the src/libpicviz/bindings directory. And the frontend is available in src/frontend.

To play with the first sample, you can run:
./picviz-gui.py ../../samples/test1.pcv

And when moving the mouse on the line, you will see the information:

Now please, before the upcoming 0.1 release, I would need testings and feedback, so please do!

Happy graphing!

Quote of the day^Hlast week

Sebastien Tricaud — Tue, 15 Jul 2008 20:26:00 +0000

Well we have a better security track record than Red Hat, we do that by focusing very hard on security -- Mark Shuttleworth

Sure, keep focusing on what Redhat is actually doing!

EuSecWest

Sebastien Tricaud — Mon, 19 May 2008 22:06:00 +0000

I will be in London in two days attending EuSecWest. If you want to share a beer and talk about IDS, Prelude, NuFW or music, feel free to drop me a mail!

PIG - Prelude IDMEF Grapher

Sebastien Tricaud — Thu, 08 May 2008 15:46:00 +0000

The Prelude IDMEF Grapher written to illustrate one aspect of Intrusion Detection Systems for the conference on the subject at CanSecWest this year is getting some attention.

Upon Raffy's request, I added to the excellent Secviz.org the generated graphs from three well-known scanners: Retina, Saint and Nessus.

I attacked my Prelude IDS machine which has two agents: Snort and Prelude LML. Those agents generate IDMEF alerts and PIG connects to the Prelude Manager to listen to any IDMEF event received.

With the power of Python+QT, in one hour I got the code up and running.

If you want to read what Ron Gula from Tenable say about it, you can read his blog post about PIG.

Right now pig's code must be ported to the recent additions from Yoann on top of what Pierre and I wrote to get Prelude easy bindings working. The merge will happen very soon with trunk and then PIG will be improved.

[RELEASE] Wolfotrack 1.0

Sebastien Tricaud — Sun, 27 Apr 2008 15:22:00 +0000

To put an end to a teaser.

Along with Laurent and Victor, we've written a Netfilter connection tracking manipulation tool based on the fabulous Wolfenstein 3d game.

You need only two dependencies: SDL library and libnetfilter_conntrack.

Description from the release notes:

Tears were flowing from our bellowed
Administrators out there.

Connection tracking is not always easy,
hence Wolfotrack, the conntrack killer that
aims to reduce the firewall use difficulty
that many people complained about for years.
This software makes this time gone! We are now
enhancing netfilter at the user level.

The idea is simple: with statefull firewall such as Netfilter, the Linux kernel firewall, connection states are kept in memory and allow you to use this simple rule to only allow answers to a previously initiated connection:
# iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

Because of the great work put by Netfilter core team into nfnetlink, and especially with the nfnetlink_conntrack socket, it is made trivial to grab the information Netfilter has with any connection state.

For example this code registers the callback function that is then used to set the players connection:

void ct_list_create(void)
{
        int ret;
        u_int8_t family = AF_INET;

        h = nfct_open(CONNTRACK, 0);
        if (!h) {
                perror("nfct_open error: Oh my god! this is terrible! you cannot kill conntracks out from Netfilter!!");
                return;
        }
        nfct_callback_register(h, NFCT_T_ALL, ct_cb, NULL);
        ret = nfct_query(h, NFCT_Q_DUMP, &family);
        if ( ret == -1 ) {
                exit(EXIT_FAILURE);
        }
}

The, the callback prototype is:

int ct_cb(enum nf_conntrack_msg_type type,
                struct nf_conntrack *ct,
                void *data)

And to set the source/destination ip and port out from the nf_conntrack structure:

        if (nfct_get_attr_u8(ct, ATTR_ORIG_L3PROTO) == AF_INET) {
                ip_src = nfct_get_attr_u32(ct, ATTR_ORIG_IPV4_SRC);
                saddr = strdup(inet_ntoa(ip_src));
                ip_dst = nfct_get_attr_u32(ct, ATTR_ORIG_IPV4_DST);
                daddr = strdup(inet_ntoa(ip_dst));
                port_src = nfct_get_attr_u16(ct, ATTR_ORIG_PORT_SRC);
                sport = strdup(port_ntoa(port_src));
                port_dst = nfct_get_attr_u16(ct, ATTR_ORIG_PORT_DST);
                dport = strdup(port_ntoa(port_dst));
        }

Finally, when you kill the actor, we run the following function:

void ct_remove_from_id(int id)
{
        if (ct_list_get(id))
                nfct_query(h, NFCT_Q_DESTROY, ct_list_get(id)->ct);
}

And that's all for the netfilter code (modulo a few tricks to set a connection tracking to a non-dead actor, ..). And here begins the Carmack magic...

I have a high respect for this guy, he started great games that I enjoyed playing in my childhood and I am still amazed by this:
i = 0x5f3759df - ( i >> 1 );

Yes, this is the famous magic constant to calculate the square root of a number with NO loop of any sort.

Putting the hands in the code, there was no PrintXY, so we wrote the function:

void US_PrintXY(char *str, word X, word Y)
{
    char c, *se, *s, *sz = strdup(str);
    word w, h;
    s = sz;
    
    while (*s)
    {
        se = s;
        while ((c = *se) && (c != '\n'))
            se++;
        *se = '\0';

        USL_MeasureString(s,&w,&h);
        px = X;
        py = Y;
        USL_DrawString(s);

        s = se;
        if (c)
        {
            *se = c;
            s++;

            X = WindowX;
            Y += h;
        }
        else
            X += w;
    }
    
    px = X;
    py = Y;
    
    free(sz);
}

and in wl_draw.c, to draw the text about connection trackings, we need to have the actor in our visual spot:

        if (*visspot
        || (*(visspot-1) && !*(tilespot-1))
        || (*(visspot+1) && !*(tilespot+1))
        || (*(visspot-65) && !*(tilespot-65))
        || (*(visspot-64) && !*(tilespot-64))
        || (*(visspot-63) && !*(tilespot-63))
        || (*(visspot+65) && !*(tilespot+65))
        || (*(visspot+64) && !*(tilespot+64))
        || (*(visspot+63) && !*(tilespot+63)))

and there while browsing the linked list of every object, we need to know if this actor is not dead, so we added this function:

int ActorDead(objtype *obj)
{
        int retval = 0;

        switch(obj->state) {
                case s_grddie4:
                        retval = 1;
                        break;
                default:
                        retval = 0;
        }
        return retval;
}

and then, in the code we use it:
if ( ( obj->obclass == guardobj ) && ( ! ActorDead(obj)) ) {

ok, shame on me, we put the connection tracking only to guards. But if you want to improve the code, this is what you must patch.

And then, if we get an appropriate connection tracking object, we run:

source = malloc(strlen(entry->saddr) +
                strlen(":") +
                strlen(entry->sport) + 1);
target = malloc(strlen(entry->daddr) +
                strlen(":") +
                strlen(entry->dport) + 1);
sprintf(source, "%s:%s", ct_list_get(obj->id)->saddr, ct_list_get(obj->id)->sport );
sprintf(target, "%s:%s", ct_list_get(obj->id)->daddr, ct_list_get(obj->id)->dport );

SETFONTCOLOR(68, BKGDCOLOR);
US_PrintXY(source, 30, 20);
US_PrintXY(target, 30, 30);
SETFONTCOLOR(TEXTCOLOR,BKGDCOLOR);

free(source);
free(target);

Then, we go into the function KillActor (objtype *ob) and if the object is a guard, we run the killing function:
ct_remove_from_id(ob->id);

And this is it! So in summary:

That was fun to do
The Wolf3d source code is crystal clear: I have never looked for hours where this or that function was. I am really amazed by the work done by the ID software team back in the early '90. Everything is very logical and I am not involved in video games in any way, so there is a lot of things I don't know
We need people to improve now. Please go to the Wolfotrack project page and download, send patches etc..

And congrats to Laurent and Victor, that was fun working in team on this kind of project

PF vs Netfilter

Sebastien Tricaud — Fri, 25 Apr 2008 21:42:00 +0000

Morceau choisi de l'interview d'Eric :

Bon, c’est sur qu’il vaut peut-être mieux avoir authpf et être attaquable par IP spoofing plutôt que de pouvoir implémenter une solution résistante comme NuFW

The best way to manage your firewall (teaser)

Sebastien Tricaud — Thu, 24 Apr 2008 21:02:00 +0000

Monday

...is software release day

Do NOT miss the Netfilter users mailing list

Once upon a time, administrators struggled to manage their firewall. That time will be in a few years, just like what happened to the city of Paris: Paris is considered to be the world's most romanticand seizable city. And the same thing will happen to firewall: they will finally be usable for normal administrators.

Stay tuned!

Is your Opera browser vulnerable ?

Sebastien Tricaud — Tue, 08 Apr 2008 22:05:00 +0000

Do you want to know if you are using a vulnerable version of Opera ?

Advisory: Newsfeed prompt can cause Opera to execute arbitrary code

Advisory: Resized canvas patterns can cause Opera to execute arbitrary code

Add the following Snort signature:

alert tcp any any -> any $HTTP_PORTS (msg:"PVR - Opera version that can be exploited by malicious people to conduct cross-site scripting attacks, disclose sensitive information, or to bypass certain security restrictions"; flags:PA; flow:to_server,established; pcre:"/User-Agent: Opera/9.2[0-6]/"; reference:url,www.opera.com/support/search/view/881/; reference:url,www.opera.com/support/search/view/882/; priority:1; sid:200804032; rev:2;)

You can of course get this signatures in the Signatures.NU project Snort Passive Vulnerabilities Rulesets (PVR).

Intrusion Detection Systems Correlation: a Weapon of Mass Investigation

Sebastien Tricaud — Mon, 07 Apr 2008 03:37:00 +0000

The paper I co-wrote with Pierre is now available:

Abstract. This paper describes how correlation can be used to reduce false positives, discover new attacks and fight the evasion of intrusion detection systems. Events from different sources, network-based, host-based and others, are used to increase the accuracy of alerts and attacks understanding. A complete solution based on the Prelude IDS framework and the Intrusion Detection Message Exchange Format (IDMEF) standard is proposed, using Voice over IP (VoIP) as an example. Taking advantage of existing products in a hierarchical manner renders more efficient the extraction of relevant security issues. We also propose an algorithm to use correlation results to lower the amount of work needed on sensors, to concentrate on higher-level attack detection.

Key words: IDS, Hybrid IDS, IDMEF, Prelude, Correlation, Management, Assessment, Visualization

Visualisation 3D d'évènements de sécurité

Sebastien Tricaud — Thu, 03 Apr 2008 23:24:00 +0000

Ce mois-ci j'ai un article dans le magazine programmez. Vous pouvez acheter le magazine pour lire notre article, le voir coupé en deux et avec des altérations non desirées par leurs auteurs, ou bien le lire sous sa forme originale et complète ici.

Snort PVR for latest CUPS vulnerabilities

Sebastien Tricaud — Thu, 03 Apr 2008 09:02:00 +0000

Following recent vulnerabilities in the CUPS server, I created two signatures alerting you if your server is vulnerable on the two following CVE:

CVE-2008-0047: Heap-based buffer overflow in the cgiCompileSearch function in CUPS 1.3.5, and other versions including the version bundled with Apple Mac OS X 10.5.2, when printer sharing is enabled, allows remote attackers to execute arbitrary code via crafted search expressions.
CVE-2008-0882: Double free vulnerability in the process_browse_data function in CUPS 1.3.5 allows remote attackers to cause a denial of service (daemon crash) and possibly execute arbitrary code via crafted UDP Browse packets to the cupsd port (631/udp), related to an unspecified manipulation of a remote printer. NOTE: some of these details are obtained from third party information.

These signatures are:

alert tcp any 631 -> any any (msg:"PVR - CUPS Heap-based buffer overflow in the cgiCompileSearch function"; flags:PA; flow:established; content:"Server: CUPS/1.3.5"; reference:cve,2008-0047; sid:200804021; rev:2;)
alert udp any 631 -> any any (msg:"PVR - CUPS Double free vulnerability in the process_browse_data function"; flags:PA; flow:established; content:"Server: CUPS/1.3.5"; reference:cve,2008-0882; sid:200804031; rev:1;)

And are of course available from the snort Passive Vulnerability Rulesets from the Signatures.NU project:

svn co http://svn.signatures.nu/snort/pvr/unstable pvr