Difference: LibLogNorm (2 vs. 3)

Revision 32011-03-16 - ChampClark

Line: 1 to 1
 
META TOPICPARENT name="WebTopicList"
Changed:
<
<
http://www.liblognorm.com/help/first-steps-using-liblognorm/
>
>
Installing / Configuring liblognorm.
 
Changed:
<
<
Coming soon!
>
>

Intro to liblognorm

"liblognorm" is a log normalization library. That is, you can feed liblognorm a "log message" (think syslog or event log) and it can extract useful information from the message. liblognorm was created by Rainer Gerhards of "Rsyslog" fame in mid-2010. It uses a simple rule syntax to extract useful information from a log message. Actually, a better way to think about it is that it uses a "masking" approach to extract information. "liblognorm" is based of the Mitre Common Event Expression Language which has not been standardized yet. While testing with Sagan, we've found liblognorm to be extremely stable, but it should be noted that since CEE is not standardized yet and it's a fairly young project, liblognorm is expected to go through changes making it a "moving target".

For a simple example of how liblognorm works, let's examine the following log message:

Invalid user bobuser from 192.168.0.1 on port 14121

Even though this is a simple example, we can already see some information we might want to extract using liblognorm. The username, TCP/IP address and TCP port number in particular. To do this, we can create a simple "rule base" for liblognorm to use to extract this information. For example:

Invalid user %username:word% from %src-ip:ipv4% on port %src-port:number%

Sagan uses liblognorm to extract information from log messages in order to better correlate the events. Sagan also has other methods of parsing, such as "parse_ip" or "parse_port", but liblognorm is by far the best at extracting useful information. Sagan rules ship with a standard rules and liblognorm "rulebase" files. These files end with the extension of .rulebase and are intended for liblognorm use. Sagan only uses liblognorm in certain cases, where rulebases or "masks" have been created from known log events to correlate information. That is, Sagan doesn't use liblognorm against every incoming log line because sometimes the log line might not contact information we need. In some dynamic log messages, it's better to use the "parse_ip" or "parse_port" flags within a rule.

You can certainly disable liblognorm at compile time, but it's advised against as liblognorm adds a lot of functionality to Sagan.

You might also want to check your operating system distribution repository for the availability of "liblognorm". For example, I know Debian has liblognorm packages available.

Compiling and installing liblognorm from the tar ball.

First, you'll need to download three tar balls. They are:

libestr
libee
liblognorm

Once downloaded, starting with libestr. The order of compiling is important, as each library depends on another....

$ tar -zxvpf libestr-{version}.tar.gz
$ cd libestr-{version}
$ ./configure --libdir=/usr/lib --includedir=/usr/include
$ make
$ sudo make install

Next, do libee .....

$ tar -zxvpf libee-{version}.tar.gz
$ cd libee-{version}
$ ./configure --libdir=/usr/lib --includedir=/usr/include
$ make
$ sudo make install

Finally the main library, liblognorm.....

$ tar -zxvpf liblognorm-{version}.tar.gz
$ cd liblognorm-{version}
$ ./configure --libdir=/usr/lib --includedir=/usr/include
$ make
$ sudo make install

Once you completed these steps, you can ./configure Sagan with liblognorm support. By re-running the Sagan ./configure, it'll detect that liblognorm is installed and use it.

Compiling and installing liblognorm from the 'git' tree.

Since liblognorm is based on CEE and can be considered a moving target, you might want to consider using the "git" source tree. The advantage to this is that you can take advantage of any new features and/or bug fixes. During our testing with Sagan and liblognorm, we've found it extremely stable. However, using the 'git' resource has the advantage of keeping up with Rainer's latest liblognorm work.

In order to install from the 'git' resource, open a terminal and enter these commands in this order:

$ git clone git://git.adiscon.com/git/libestr.git

switch to the new folder libestr

$ autoreconf -vfi
$ ./configure --libdir=/usr/lib --includedir=/usr/include
$ make
$ sudo make install

leave that folder and repeat this step for libee

$ git clone git://git.adiscon.com/git/libee.git

switch to the new folder libee

$ autoreconf -vfi
$ ./configure --libdir=/usr/lib --includedir=/usr/include
$ make
$ sudo make install

leave that folder and repeat this step again for liblognorm

$ git clone git://git.adiscon.com/git/liblognorm.git

switch to the new folder liblognorm

$ autoreconf -vfi
$ ./configure --libdir=/usr/lib --includedir=/usr/include
$ make
$ sudo make install

This above was largely taken from http://www.liblognorm.com/help/first-steps-using-liblognorm/.

Once you completed these steps, you can ./configure Sagan with liblognorm support. By re-running the Sagan ./configure, it'll detect that liblognorm is installed and use it.

  -- ChampClark - 2010-12-10 \ No newline at end of file
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2014 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback