The policy/frameworks/intel/seen bro scripts have a memory leak. On my moderately busy Bro installation I am leaking about a gig of memory a day per worker process with the Intel framework enabled. I can replicate by adding the following to the local.bro default script and then running through a small PCAP with primarily dns, dhcp and syslog traffic.
{{
@load policy/frameworks/intel/seen
redef Intel::read_files += {
"/usr/local/bro/spool/domain_suspicious.txt",
};
}}
The intel file is in the following format, here's a few sample lines. It is generated automatically by CIF:
{{
#fields indicator indicator_type meta.source meta.desc meta.url meta.cif_impact meta.cif_severity meta.cif_confidence
mete-tools.biz Intel:OMAIN CIF - need-to-know spammed domain http://www.spamhaus.org/query/dbl?domain=mete-tools.biz (public) - - 95
rttvxygkmwlqmq.net Intel:OMAIN CIF - need-to-know spammed domain http://www.spamhaus.org/query/dbl?domain=rttvxygkmwlqmq.net (public) - - 95
podserveruho.com Intel:OMAIN CIF - need-to-know spammed domain http://www.spamhaus.org/query/dbl?domain=podserveruho.com (public) - - 95
wwfcogdgntlxw.biz Intel:OMAIN CIF - need-to-know spammed domain http://www.spamhaus.org/query/dbl?domain=wwfcogdgntlxw.biz (public) - - 95
}}
I compiled bro with gperftool debug support and followed the instructions here: http://www.bro.org/development/howtos/leaks.html. (Note, the instructions are wrong on the flags for ./configure, you need to add --enable-perftools-debug to get the -m option for bro)
Here's the output from pprof top after running a PCAP trace with 10,000 packets. Running traces with more packets show a greater number of lost objects in the same code locations.
{{
pprof bin/bro "/tmp/bro.24541.net_run-end.heap" --inuse_objects --lines --heapcheck --edgefraction=1e-10 --nodefraction=1e-10
Using local file bin/bro.
Using local file /tmp/bro.24541.net_run-end.heap.
Welcome to pprof! For help, type 'help'.
(pprof) top
Total: 4295 objects
2150 50.1% 50.1% 2150 50.1% AsciiFormatter:arseValue /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:186
2141 49.8% 99.9% 2141 49.8% copy_string /usr/src/bro-2.2/src/util.cc:155
2 0.0% 100.0% 2 0.0% re_alloc /usr/src/bro-2.2/build/src/re-scan.cc:2287
1 0.0% 100.0% 1 0.0% RE_parse /usr/src/bro-2.2/build/src/re-parse.y:110
1 0.0% 100.0% 1 0.0% RE_parse /usr/src/bro-2.2/build/src/re-parse.y:133
0 0.0% 100.0% 2141 49.8% AsciiFormatter:arseValue /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:195
0 0.0% 100.0% 4 0.1% Connection::NextPacket /usr/src/bro-2.2/src/Conn.cc:259
0 0.0% 100.0% 4 0.1% NetSessions:ispatchPacket /usr/src/bro-2.2/src/Sessions.cc:189
0 0.0% 100.0% 4 0.1% NetSessions:oNextPacket /usr/src/bro-2.2/src/Sessions.cc:709
0 0.0% 100.0% 4 0.1% NetSessions::NextPacket /usr/src/bro-2.2/src/Sessions.cc:247
}}
Red Hat Enterprise Linux Server release 6.5
I ran some more checks and the memory leak is only based on the size and frequency of the files loaded through Intel::read_files, the size and content of the trace files tested doesn't seem to make much difference in the final number of lost objects.
I have confirmed this leak, it seems to be an issue with the input framework. We'll be fixing it ASAP. Thanks for tracking it down.
Seth and me tracked down the root of the leak - the input manager uses the length of the root-record while deleting values instead of the length of the unrolled record.
I will fix this in a few hours (well, after re-checking it really fixes it), thank you very much for the very thorough bug-report.
verified patch is in topic/bernhard/ticket1103
I applied the patch against 2.2 and verified that it fixed the memory leak on my system. Thank you!