Zeek XDP Filtering

Uses XDP in order to "shunt" traffic - that way, Zeek doesn't waste its time analyzing packets that won't get any useful information.

You can use the provided configuration that is similar in functionality to bro-react:

@load xdp/shunt/bulk

This provides a few configuration knobs, but it should shunt connections that are deemed "too large" and unshunt it when a certain time elapses without the connection terminating.

The XDP program will still forward certain TCP packets, namely FIN, RST, SYN, and ACK packets. This way, Zeek can tell when a connection is complete and properly terminate it.

Manual Shunting

You may also choose to manually shunt connections based on your own criteria. You can use the provided BIFs for that. For example, this shunts encrypted sessions, then unshunts when it sees a FIN/RST TCP packet:

@load xdp
@load xdp/shunt/conn_id

global xdp_prog: opaque of XDP::Program;

event zeek_init()
	{
	local opts: XDP::ShuntOptions = [
		$attach_mode=XDP::SKB, # SKB is more for testing, NATIVE is better
		$conn_id_map_max_size=131072, # A bit over the max number of shunted connections
		$ip_pair_map_max_size=1, # We can't remove the map, so make it very small
	];
	xdp_prog = XDP::start_shunt(opts);
	}

event connection_state_remove(c: connection)
	{
	XDP::Shunt::ConnID::unshunt(xdp_prog, XDP::conn_id_to_canonical(c$id));
	}

event XDP::Shunt::ConnID::unshunted_conn(cid: XDP::canonical_id,
    stats: XDP::ShuntedStats)
	{
	assert stats$present;
	print fmt("Unshunted connection from %s:%d<->%s:%d. Transmitted %d bytes and %d packets. Last packet was at %s.",
	    cid$ip1, cid$ip1_port, cid$ip2, cid$ip2_port, stats$bytes_from_1 +
	    stats$bytes_from_2, stats$packets_from_1 + stats$packets_from_2,
	    stats$timestamp);
	}

event XDP::Shunt::ConnID::shunted_conn(cid: XDP::canonical_id)
	{
	print fmt("Shunted connection from %s:%d<->%s:%d", cid$ip1, cid$ip1_port,
	    cid$ip2, cid$ip2_port);
	}

event ssl_established(c: connection)
	{
	XDP::Shunt::ConnID::shunt(xdp_prog, XDP::conn_id_to_canonical(c$id));
	}

event zeek_done()
	{
	XDP::end_shunt(xdp_prog);
	}

IP Pairs

Shunting with IP pairs is similar:

@load xdp
@load xdp/shunt/ip_pair

global xdp_prog: opaque of XDP::Program;

event zeek_init()
	{
	local opts: XDP::ShuntOptions = [
		$attach_mode=XDP::SKB, # SKB is more for testing, NATIVE is better
		$conn_id_map_max_size=1, # We can't remove the map, so make it very small
		$ip_pair_map_max_size=131072, # A bit over the max number of shunted connections
	];
	xdp_prog = XDP::start_shunt(opts);
	}

event connection_state_remove(c: connection)
	{
	XDP::Shunt::IPPair::unshunt(xdp_prog, [ $ip1=c$id$orig_h, $ip2=c$id$resp_h ]);
	}

event XDP::Shunt::IPPair::unshunted_pair(pair: XDP::ip_pair,
    stats: XDP::ShuntedStats)
	{
	assert stats$present;
	print fmt("Unshunted connection from %s<->%s. Transmitted %d bytes and %d packets. Last packet was at %s.",
	    pair$ip1, pair$ip2, stats$bytes_from_1 + stats$bytes_from_2,
	    stats$packets_from_1 + stats$packets_from_2, stats$timestamp);
	}

event XDP::Shunt::IPPair::shunted_pair(pair: XDP::ip_pair)
	{
	print fmt("Shunted connection from %s<->%s", pair$ip1, pair$ip2);
	}

event ssl_established(c: connection)
	{
	XDP::Shunt::IPPair::shunt(xdp_prog, [ $ip1=c$id$orig_h, $ip2=c$id$resp_h ]);
	}

event zeek_done()
	{
	XDP::end_shunt(xdp_prog);
	}

VLANs

By default, the XDP program will parse only a static number of VLAN headers, defined within the file. This should be adequate for most use cases. It will always parse the VLAN headers, but it will only use them if set with the include_vlan config option, like so:

event zeek_init()
	{
	local opts: XDP::ShuntOptions = [
		$attach_mode=XDP::SKB,
		$conn_id_map_max_size=131072,
		$ip_pair_map_max_size=1,
		$include_vlan=T, # Added this!
	];
	xdp_prog = XDP::start_shunt(opts);
	}

Then, the "canonical" ID does not add the VLAN by default, so the user has to add it. You can ask Zeek to bring the VLANs in the conn_id by loading a policy script:

@load policy/frameworks/conn_key/vlan_fivetuple

Then add the information to the canonical ID when shunting and unshunting:

event connection_state_remove(c: connection)
	{
	local new_id = XDP::conn_id_to_canonical(c$id);

	if ( c$id$ctx?$vlan )
		new_id$outer_vlan_id = c$id$ctx$vlan;

	if ( c$id$ctx?$inner_vlan )
		new_id$inner_vlan_id = c$id$ctx$inner_vlan;

	XDP::Shunt::ConnID::unshunt(xdp_prog, new_id);
	}

# ...

event ssl_established(c: connection)
	{
	local new_id = XDP::conn_id_to_canonical(c$id);

	if ( c$id$ctx?$vlan )
		new_id$outer_vlan_id = c$id$ctx$vlan;

	if ( c$id$ctx?$inner_vlan )
		new_id$inner_vlan_id = c$id$ctx$inner_vlan;

	XDP::Shunt::ConnID::shunt(xdp_prog, new_id);
	}

A VLAN of 0 is considered the same as a VLAN with no ID for the purposes of shunting.

Internals

This plugin uses XDP (eXpress Data Path) in order to filter traffic before it reaches user applications, like Zeek. This is filtering purely on the network device that Zeek sees traffic from.

The XDP program itself simply checks if a certain IP pair or 5-tuple are in the "shunting map." This map is managed entirely from user space, that is, the Zeek program. You may also, optionally, make it VLAN-aware (see above).

Any shunted connections keep state about certain statistics, such as the last packet seen and how many bytes/packets were seen from each direction. This may help the user determine how effective the shunting was, or simply unshunt connections after a certain time elapsed without any traffic.

TCP control packets, like FIN and RST, are forwarded to Zeek regardless of content. This allows Zeek to determine that the connection was terminated by a FIN or RST rather than by timeout.