Broker's integration in Bro's main/run loop

Description

  • There's a cost to Broker queues being idle. Whenever Broker gets a chance to process messages, it looks for updates to all connections/message-queues/data-stores. That involves sending synchronous messages between actors, and for empty queues, it just gets back an empty deque object it needs to destroy.

  • Broker queues integrate into Bro's run loop by exposing a file descriptor that's ready when the queue is non-empty. Users have the capability of adding arbitrary numbers of queues at run-time (e.g. they can freely add subscriptions to any amount of logs, events, etc.). Relying on select() may become a bottleneck if someone has hundreds of Broker queues, or could possibly break on some systems if FD_SETSIZE is limited to 1024.

Ideas on how to fix:

  • Improve Bro's main run loop and dedicate an IOSource to each Broker queue (instead of sharing a single IOSource like they do now). There might be several things that could be tweaked in the main run loop, but at a minimum, epoll()/kqueue() could alternatively replace select(). Could also think about using something like libev (http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod) to abstract what particular polling backend is used. Might even be able to use libev's timers to fix how Bro's timers are currently coupled w/ there being an active IOSource consistently driving time forward.

  • Move the draining of Broker queues completely off to their own threads. This maybe is adding a bit too much complexity (Broker/CAF uses threads for queues, then Bro would use more threads just to talk to those other threads...). Since CAF becomes a requirement, it may be simpler to start replacing/allowing some areas of Bro's threading to be done w/ CAF actors. And then if Broker exposed an optional API to talk directly w/ CAF actors, the integration w/ Bro may actually become more straightforward.

And those ideas don't have to be mutually exclusive.

Environment

None

Assignee

Jon Siwek

Reporter

Jon Siwek

Labels

None

External issue ID

None

Components

Priority

Normal
Configure