Investigate replacing libmagic w/ signatures for file identificaiton

Description

I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules.

Environment

None

Activity

Show:
Jon Siwek
March 6, 2014, 8:49 PM

I also forgot to mention another improvement of the signature approach over libmagic is that a file is no longer limited to matching a single MIME type. One can now programmatically get at the full list of signature matches along with a value indicating the "strength" of the match.

Seth Hall
March 6, 2014, 8:52 PM

I was already working on this branch, so I'll go ahead and claim it for a day or so while I play around. It looks really awesome though.

Jon Siwek
March 24, 2014, 3:47 PM

Seth do you have any feedback in these areas:

  • Notice anything missing from script-layer support of file-type detection? The only difference should be all matches are available instead of just one, so I don't expect any issue, but asking just in case.

  • Notice any problems with the file-magic signature grammar?

  • Are the default set of file-magic rules adequate or is there something that definitely needs work before merging (opposed to making iterative improvements later on) ?

If no problems, I'll set this to a merge request.

Seth Hall
March 24, 2014, 4:28 PM

Everything looked ok to me when I was playing with it. I think it's probably ready to be merged.

Jon Siwek
March 25, 2014, 5:57 PM

merge-ready version is still topic/jsiwek/file-signatures in bro, 3rdparty, bro-testing, and bro-testing-private

Assignee

Robin Sommer

Reporter

Jon Siwek

Labels

None

External issue ID

None

Components

Fix versions

Affects versions

Priority

Normal
Configure