bro segfaults at analyzer::mime::MIME_Entity::ParseFieldParameters

Description

bro worker segfaults occurred from time to time after upgrade to bro 2.4-78 . Looks like the problem rise in analyzer::mime::MIME_Entity:arseFieldParameters (/usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:126). A couple of core listings follows:
Core was generated by `/usr/local/bin/bro -i zc:99@2 -U .status -p broctl -p broctl-live -p local -p w'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 analyzer::mime::MIME_Entity:arseFieldParameters (this=this@entry=0x8aae540, len=16, len@entry=27, data=0x2447faec "(UploadBoundary)", data@entry=0x2447fae1 "; boundary=(UploadBoundary)")
at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:126
126 static data_chunk_t get_data_chunk(BroString* s)
(gdb) backtrace
#0 analyzer::mime::MIME_Entity:arseFieldParameters (this=this@entry=0x8aae540, len=16, len@entry=27, data=0x2447faec "(UploadBoundary)", data@entry=0x2447fae1 "; boundary=(UploadBoundary)")
at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:126
#1 0x0000000000769f7c in analyzer::mime::MIME_Entity:arseContentTypeField (this=this@entry=0x8aae540, h=h@entry=0x521ddc0) at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:799
#2 0x000000000076a1d1 in analyzer::mime::MIME_Entity:arseMIMEHeader (this=this@entry=0x8aae540, h=h@entry=0x521ddc0) at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:763
#3 0x000000000076b638 in analyzer::mime::MIME_Entity::FinishHeader (this=this@entry=0x8aae540) at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:735
#4 0x000000000076b821 in analyzer::mime::MIME_Entity::NewHeader (this=0x8aae540, len=13, data=0x1704a3c0 "Host: fegi.ru") at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:699
#5 0x0000000000721490 in analyzer::http::HTTP_Analyzer:eliverStream (this=0xbd9f080, len=13, data=0x1704a3c0 "Host: fegi.ru", is_orig=<optimized out>)
at /usr/src/other/bro/src/analyzer/protocol/http/HTTP.cc:1038
#6 0x00000000007f0ded in analyzer::tcp::ContentLine_Analyzer:oDeliverOnce (this=this@entry=0x14fbe090, len=<optimized out>, len@entry=84, data=<optimized out>,
data@entry=0xcd56528 "Host: fegi.ru\r\nContent-Length: 185\r\nExpect: 100-continue\r\nConnection: Keep-Alive\r\n\r\n") at /usr/src/other/bro/src/analyzer/protocol/tcp/ContentLine.cc:258
#7 0x00000000007f0fbb in analyzer::tcp::ContentLine_Analyzer:oDeliver (this=0x14fbe090, len=84,
data=0xcd56528 "Host: fegi.ru\r\nContent-Length: 185\r\nExpect: 100-continue\r\nConnection: Keep-Alive\r\n\r\n") at /usr/src/other/bro/src/analyzer/protocol/tcp/ContentLine.cc:200
#8 0x00000000007f07b0 in analyzer::tcp::ContentLine_Analyzer:eliverStream (this=0x14fbe090, len=<optimized out>,
data=0xcd563c0 "POST /wp-content/themes/ProjectTheme/lib/upload_main/upload.php?folder=/wp-content/uploads/ HTTP/1.1\r\nReferer: http://fegi.ru/wp-content/themes/ProjectTheme/lib/upload_main/upload.php?folder=/wp-conte"..., is_orig=<optimized out>) at /usr/src/other/bro/src/analyzer/protocol/tcp/ContentLine.cc:108
#9 0x0000000000861216 in analyzer::Analyzer::NextStream (this=0x14fbe090, len=444,
data=0xcd563c0 "POST /wp-content/themes/ProjectTheme/lib/upload_main/upload.php?folder=/wp-content/uploads/ HTTP/1.1\r\nReferer: http://fegi.ru/wp-content/themes/ProjectTheme/lib/upload_main/upload.php?folder=/wp-conte"..., is_orig=<optimized out>) at /usr/src/other/bro/src/analyzer/Analyzer.cc:245
#10 0x00000000008619a6 in analyzer::Analyzer::ForwardStream (this=0x14ea0000, len=444,
data=0xcd563c0 "POST /wp-content/themes/ProjectTheme/lib/upload_main/upload.php?folder=/wp-content/uploads/ HTTP/1.1\r\nReferer: http://fegi.ru/wp-content/themes/ProjectTheme/lib/upload_main/upload.php?folder=/wp-conte"..., is_orig=<optimized out>) at /usr/src/other/bro/src/analyzer/Analyzer.cc:331
#11 0x00000000007efb49 in analyzer::tcp::TCP_Reassembler:eliverBlock (this=this@entry=0xc6d7800, seq=seq@entry=1, len=len@entry=444,
data=0xcd563c0 "POST /wp-content/themes/ProjectTheme/lib/upload_main/upload.php?folder=/wp-content/uploads/ HTTP/1.1\r\nReferer: http://fegi.ru/wp-content/themes/ProjectTheme/lib/upload_main/upload.php?folder=/wp-conte"...) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:650
#12 0x00000000007efe79 in analyzer::tcp::TCP_Reassembler::BlockInserted (this=0xc6d7800, start_block=<optimized out>) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:396
#13 0x00000000007ef9cc in analyzer::tcp::TCP_Reassembler:ataSent (this=0xc6d7800, t=<optimized out>, seq=<optimized out>, len=<optimized out>, len@entry=444, data=<optimized out>,
data@entry=0x7f5b768985b6 <error: Cannot access memory at address 0x7f5b768985b6>, replaying=replaying@entry=true) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:495
#14 0x00000000007ee341 in analyzer::tcp::TCP_Endpoint:ataSent (this=this@entry=0x710d620, t=<optimized out>, seq=seq@entry=1, len=444, caplen=444,
data=0x7f5b768985b6 <error: Cannot access memory at address 0x7f5b768985b6>, ip=ip@entry=0x7ffcb14c4f90, tp=tp@entry=0x7f5b768985a2)
at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP_Endpoint.cc:207
#15 0x00000000007eba12 in DeliverData (flags=..., is_orig=<optimized out>, rel_data_seq=1, endpoint=0x710d620, tp=0x7f5b768985a2, ip=0x7ffcb14c4f90, caplen=<optimized out>, len=<optimized out>,
data=<optimized out>, t=<optimized out>, this=0x14ea0000) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP.cc:982
#16 analyzer::tcp::TCP_Analyzer:eliverPacket (this=0x14ea0000, len=444, data=0x7f5b768985b6 <error: Cannot access memory at address 0x7f5b768985b6>, is_orig=<optimized out>, seq=<optimized out>,
ip=0x7ffcb14c4f90, caplen=444) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP.cc:1382
#17 0x00000000008610c2 in analyzer::Analyzer::NextPacket (this=0x14ea0000, len=464, data=0x7f5b768985a2 <error: Cannot access memory at address 0x7f5b768985a2>, is_orig=<optimized out>,
seq=18446744073709551615, ip=0x7ffcb14c4f90, caplen=464) at /usr/src/other/bro/src/analyzer/Analyzer.cc:222
#18 0x000000000056979d in Connection::NextPacket (this=this@entry=0x1d1b6540, t=t@entry=1439902857.1053071, is_orig=is_orig@entry=1, ip=ip@entry=0x7ffcb14c4f90, len=len@entry=464,
caplen=caplen@entry=464, data=@0x7ffcb14c4e08: 0x7f5b768985a2 <error: Cannot access memory at address 0x7f5b768985a2>, record_packet=<optimized out>, record_content=<optimized out>,
pkt=<optimized out>, pkt@entry=0x2821530) at /usr/src/other/bro/src/Conn.cc:260
#19 0x00000000006038a0 in NetSessions:oNextPacket (this=this@entry=0x2d603c0, t=t@entry=1439902857.1053071, pkt=pkt@entry=0x2821530, ip_hdr=ip_hdr@entry=0x7ffcb14c4f90,
encapsulation=encapsulation@entry=0x0) at /usr/src/other/bro/src/Sessions.cc:735
#20 0x0000000000604824 in NetSessions::NextPacket (this=0x2d603c0, t=t@entry=1439902857.1053071, pkt=pkt@entry=0x2821530) at /usr/src/other/bro/src/Sessions.cc:207
#21 0x00000000005d456f in net_packet_dispatch (t=1439902857.1053071, pkt=pkt@entry=0x2821530, src_ps=src_ps@entry=0x2821500) at /usr/src/other/bro/src/Net.cc:273
#22 0x0000000000834539 in iosource:ktSrc:rocess (this=0x2821500) at /usr/src/other/bro/src/iosource/PktSrc.cc:265
#23 0x00000000005d4a0f in net_run () at /usr/src/other/bro/src/Net.cc:321
#24 0x00000000005346dc in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/other/bro/src/main.cc:1191
---------------------------------------------------------------------------------------------------------------------
#0 analyzer::mime::MIME_Entity:arseFieldParameters (this=this@entry=0x16141d40, len=0, len@entry=11, data=0x1c0d0e9c "", data@entry=0x1c0d0e91 "; boundary=")
at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:126
#1 0x0000000000769f7c in analyzer::mime::MIME_Entity:arseContentTypeField (this=this@entry=0x16141d40, h=h@entry=0x1a46c740) at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:799
#2 0x000000000076a1d1 in analyzer::mime::MIME_Entity:arseMIMEHeader (this=this@entry=0x16141d40, h=h@entry=0x1a46c740) at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:763
#3 0x000000000076b638 in analyzer::mime::MIME_Entity::FinishHeader (this=this@entry=0x16141d40) at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:735
#4 0x000000000076b821 in analyzer::mime::MIME_Entity::NewHeader (this=0x16141d40, len=175,
data=0xd0dee00 "User-Agent: Mozilla/5.0 (Linux; Android 4.1.2; s4507 Build/JZO54K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 YaBrowser/15.4.2272.3842.00 Mobile Safari/537.36")
at /usr/src/other/bro/src/analyzer/protocol/mime/MIME.cc:699
#5 0x0000000000721490 in analyzer::http::HTTP_Analyzer:eliverStream (this=0xe7c4080, len=175,
data=0xd0dee00 "User-Agent: Mozilla/5.0 (Linux; Android 4.1.2; s4507 Build/JZO54K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 YaBrowser/15.4.2272.3842.00 Mobile Safari/537.36",
is_orig=<optimized out>) at /usr/src/other/bro/src/analyzer/protocol/http/HTTP.cc:1038
#6 0x00000000007f0ded in analyzer::tcp::ContentLine_Analyzer:oDeliverOnce (this=this@entry=0xe806450, len=<optimized out>, len@entry=265, data=<optimized out>,
data@entry=0x21c2647 "User-Agent: Mozilla/5.0 (Linux; Android 4.1.2; s4507 Build/JZO54K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 YaBrowser/15.4.2272.3842.00 Mobile Safari/537.36\r\nAccept-Encoding: gzip, "...) at /usr/src/other/bro/src/analyzer/protocol/tcp/ContentLine.cc:258
#7 0x00000000007f0fbb in analyzer::tcp::ContentLine_Analyzer:oDeliver (this=0xe806450, len=265,
data=0x21c2647 "User-Agent: Mozilla/5.0 (Linux; Android 4.1.2; s4507 Build/JZO54K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 YaBrowser/15.4.2272.3842.00 Mobile Safari/537.36\r\nAccept-Encoding: gzip, "...) at /usr/src/other/bro/src/analyzer/protocol/tcp/ContentLine.cc:200
#8 0x00000000007f07b0 in analyzer::tcp::ContentLine_Analyzer:eliverStream (this=0xe806450, len=<optimized out>,
data=0x21c2580 "POST /submit HTTP/1.1\r\nHost: crash-reports.browser.yandex.net\r\nConnection: keep-alive\r\nContent-Length: 32768\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nContent-Type: multipart/form-data; boundary=\r\nU"..., is_orig=<optimized out>) at /usr/src/other/bro/src/analyzer/protocol/tcp/ContentLine.cc:108
#9 0x0000000000861216 in analyzer::Analyzer::NextStream (this=0xe806450, len=464,
data=0x21c2580 "POST /submit HTTP/1.1\r\nHost: crash-reports.browser.yandex.net\r\nConnection: keep-alive\r\nContent-Length: 32768\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nContent-Type: multipart/form-data; boundary=\r\nU"..., is_orig=<optimized out>) at /usr/src/other/bro/src/analyzer/Analyzer.cc:245
#10 0x00000000008619a6 in analyzer::Analyzer::ForwardStream (this=0xb172f20, len=464,
data=0x21c2580 "POST /submit HTTP/1.1\r\nHost: crash-reports.browser.yandex.net\r\nConnection: keep-alive\r\nContent-Length: 32768\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nContent-Type: multipart/form-data; boundary=\r\nU"..., is_orig=<optimized out>) at /usr/src/other/bro/src/analyzer/Analyzer.cc:331
#11 0x00000000007efb49 in analyzer::tcp::TCP_Reassembler:eliverBlock (this=this@entry=0x167805a0, seq=seq@entry=1, len=len@entry=464,
data=0x21c2580 "POST /submit HTTP/1.1\r\nHost: crash-reports.browser.yandex.net\r\nConnection: keep-alive\r\nContent-Length: 32768\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nContent-Type: multipart/form-data; boundary=\r\nU"...) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:650
#12 0x00000000007efe79 in analyzer::tcp::TCP_Reassembler::BlockInserted (this=0x167805a0, start_block=<optimized out>) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:396
#13 0x00000000007ef9cc in analyzer::tcp::TCP_Reassembler:ataSent (this=0x167805a0, t=<optimized out>, seq=<optimized out>, len=<optimized out>, len@entry=464, data=<optimized out>,
data@entry=0x7f9c1b006442 <error: Cannot access memory at address 0x7f9c1b006442>, replaying=replaying@entry=true) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:495
#14 0x00000000007ee341 in analyzer::tcp::TCP_Endpoint:ataSent (this=this@entry=0x4bb1fb0, t=<optimized out>, seq=seq@entry=1, len=464, caplen=464,
data=0x7f9c1b006442 <error: Cannot access memory at address 0x7f9c1b006442>, ip=ip@entry=0x7fff4034c130, tp=tp@entry=0x7f9c1b006422)
at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP_Endpoint.cc:207
#15 0x00000000007eba12 in DeliverData (flags=..., is_orig=<optimized out>, rel_data_seq=1, endpoint=0x4bb1fb0, tp=0x7f9c1b006422, ip=0x7fff4034c130, caplen=<optimized out>, len=<optimized out>,
data=<optimized out>, t=<optimized out>, this=0xb172f20) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP.cc:982
#16 analyzer::tcp::TCP_Analyzer:eliverPacket (this=0xb172f20, len=464, data=0x7f9c1b006442 <error: Cannot access memory at address 0x7f9c1b006442>, is_orig=<optimized out>, seq=<optimized out>,
ip=0x7fff4034c130, caplen=464) at /usr/src/other/bro/src/analyzer/protocol/tcp/TCP.cc:1382
#17 0x00000000008610c2 in analyzer::Analyzer::NextPacket (this=0xb172f20, len=496, data=0x7f9c1b006422 <error: Cannot access memory at address 0x7f9c1b006422>, is_orig=<optimized out>,
seq=18446744073709551615, ip=0x7fff4034c130, caplen=496) at /usr/src/other/bro/src/analyzer/Analyzer.cc:222
#18 0x000000000056979d in Connection::NextPacket (this=this@entry=0x11e52f40, t=t@entry=1439788398.623282, is_orig=is_orig@entry=1, ip=ip@entry=0x7fff4034c130, len=len@entry=496,
caplen=caplen@entry=496, data=@0x7fff4034bfa8: 0x7f9c1b006422 <error: Cannot access memory at address 0x7f9c1b006422>, record_packet=<optimized out>, record_content=<optimized out>,
pkt=<optimized out>, pkt@entry=0x251a870) at /usr/src/other/bro/src/Conn.cc:260
#19 0x00000000006038a0 in NetSessions:oNextPacket (this=this@entry=0x2a583c0, t=t@entry=1439788398.623282, pkt=pkt@entry=0x251a870, ip_hdr=ip_hdr@entry=0x7fff4034c130,
encapsulation=encapsulation@entry=0x0) at /usr/src/other/bro/src/Sessions.cc:735
#20 0x0000000000604824 in NetSessions::NextPacket (this=0x2a583c0, t=t@entry=1439788398.623282, pkt=pkt@entry=0x251a870) at /usr/src/other/bro/src/Sessions.cc:207
#21 0x00000000005d456f in net_packet_dispatch (t=1439788398.623282, pkt=pkt@entry=0x251a870, src_ps=src_ps@entry=0x251a840) at /usr/src/other/bro/src/Net.cc:273
#22 0x0000000000834539 in iosource:ktSrc:rocess (this=0x251a840) at /usr/src/other/bro/src/iosource/PktSrc.cc:265
#23 0x00000000005d4a0f in net_run () at /usr/src/other/bro/src/Net.cc:321
#24 0x00000000005346dc in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/other/bro/src/main.cc:1191

Environment

2xXeon E5540, 64GB RAM, Linux 3.18.11, PF_RING 6.0.3 ZC (zbalance_ipc), bro cluster

Status

Assignee

Unassigned

Reporter

Alexander Zatserkovnyy

Labels

External issue ID

None

Components

Affects versions

git/master

Priority

Normal
Configure