forget_socket does not work with a physical mirror port

[ewust]

A server has a default interface (eth0 10.0.0.5/24 with default gw 10.0.0.1), and a tap interface (eth1 192.168.0.5/24 with no gw) which is receiving traffic on a mirror port from a switch.

The server observes a flow on the tap interface, and decides to interject itself using forge_socket, and creates the requisite flows. If the server sends data through this forge_socket, the packets will leave eth0 (the default route) with correct IP/TCP parameters. However, any ACKs or responses that come back for this socket will arrive on eth1, the tap interface, for a MAC address that does not belong to either of our interfaces.

Disabling rp_filter, and/or enabling ip_forward do not seem to solve this problem, nor does setting eth1's MAC address to the expected value.

[ewust]

For using forgesocket with remote hosts, you'll want the following:

iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT

ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100

You may also want to rewrite the mac address destination to your box (using ebtables?), though I haven't found a good way to do this.