DNS_Mgr::LookupAddr does not respect DNS_FAKE

Description

None

Environment

None

Activity

Show:
Jon Siwek
March 10, 2014, 9:56 PM

topic/jsiwek/dns_fake in bro and bro-testing-private repos

Robin Sommer
March 13, 2014, 9:32 PM

Merging. I'm suggesting a further change though: what do you think about making the fake values deterministic based on the queries? E.g., BuildFakeNameResult(ip) could return "fake_<ip>" and BuildFakeAddrResult(name) an address derived from hash(name).

Advantage: now that the test-suite runs in DNS_FAKE mode, changes that affect DNS resolution wouldn't trigger diffs.

Jon Siwek
March 14, 2014, 3:34 PM

Technically, I'm not sure it's actually an advantage in terms of less test suite diffs getting triggered because I think output will always be deterministic and based on the order in which lookups are executed (fake results are available immediately, so callbacks are always executed in order and given the same fake result value across runs). If fake result values are derived from the input, test baselines still change under the same circumstances – when Bro scripts change the order in which lookup_* builtins are invoked. Or is there something else I'm not getting about it?

Practically, having the fake results derived from the input does probably make output easier to follow for a human inspector.

Robin Sommer
March 14, 2014, 4:00 PM

If just order changes, or say new lookups get introduced, the output
wouldn't change if we derive the fake results directly from the
queries. If the input changes to trigger different queries, we'd still
get diffs but maybe they would now affect only a subset of the test
run's output, not all subsequent lookups. In other words, I think this
would reduce the likelihood and volume of diffs, but it can't fiully
avoid them of course.

Practically, having the fake results derived from the input does probably make output easier to follow for a human inspector.

Ack.

Jon Siwek
March 14, 2014, 4:12 PM

we'd still get diffs but maybe they would now affect only a subset of the test
run's output, not all subsequent lookups. In other words, I think this
would reduce the likelihood and volume of diffs, but it can't fiully
avoid them of course.

Ah, I get it now and agree.

Assignee

Jon Siwek

Reporter

Justin Azoff

Labels

None

External issue ID

None

Components

Fix versions

Affects versions

Priority

Low
Configure