I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules.
I also forgot to mention another improvement of the signature approach over libmagic is that a file is no longer limited to matching a single MIME type. One can now programmatically get at the full list of signature matches along with a value indicating the "strength" of the match.
I was already working on this branch, so I'll go ahead and claim it for a day or so while I play around. It looks really awesome though.
Seth do you have any feedback in these areas:
Notice anything missing from script-layer support of file-type detection? The only difference should be all matches are available instead of just one, so I don't expect any issue, but asking just in case.
Notice any problems with the file-magic signature grammar?
Are the default set of file-magic rules adequate or is there something that definitely needs work before merging (opposed to making iterative improvements later on) ?
If no problems, I'll set this to a merge request.
Everything looked ok to me when I was playing with it. I think it's probably ready to be merged.
merge-ready version is still topic/jsiwek/file-signatures in bro, 3rdparty, bro-testing, and bro-testing-private