policy based routing?

[Stryker]

I believe policy based routing is what's best for my situation, but I have no experience in it and am not sure exactly how to set it up. Here is my situation: I have 2 network adapters, eth0 and eth1. eth0 has 64.34.37.224/27 with a gw of 64.34.37.225. eth1 has 204.15.230.0/24 with 204.15.230.1. I have them both online and working fine by just adding the default gateways to the routing table. However, whenever there's a request on eth1, the response is sent out via eth0. I wouldn't normally consider this a problem, but the link on eth0 is quite a bit more expensive to use and they're seperated for a reason.

Actually, at this point in my post, I think I got it working, but I don't wanna delete anything in case I did something wrong. Here is what I have:

Code: Select all

ip rule add from 64.34.37.224/27 table 1
ip route add default table 1 via 64.34.37.225
ip rule add from 204.15.230.0/24 table 2
ip route add default table 2 via 204.15.230.1


This appears to be working, I need to wait a while for cacti to update enough for me to see the difference. But does it look right?

[Void Main]

If you want return traffic to go out the same interface it came in on then I think that looks right. Let me know how it works out. It should be easy to test. Just fire up tcpdump/ethereal on both interfaces and connect to each ip address and see where the reply traffic goes. Otherwise, if you just want most of your return traffic to go out your less expensive interface then all you would have to do is set your default route to go out that interface as I am sure you know.

Nice article:
http://www.linuxjournal.com/article/7291

[Stryker]

it doesn't seem to work perfectly

I have to add the ip address specifically it looks like, such as:

ip rule add from 64.34.37.233/27 table 1

then it gets routed fine on that ip, but when i delete that rule, its still routed there... like it remembers even tho the rule is gone.

It'll be a pain to have to list every ip address, I will need to think of a better method I think.

edit: going to append "ip route flush cache" to the startup script, see if it helps.

[Void Main]

Do you even need policy based routing? Like I said, you can make your default route your cheap line and then add static network routes for address ranges you want to go out the expensive interface:

# route add default gw 204.15.230.1 eth1
# route add -net 12.12.0.0 netmask 255.255.0.0 gw 64.34.37.225 eth0

12.12.0.0/16 addresses will take the expensive route while everything else takes the cheap way out. Add as many static network routes as you need. That all assumes that you know which networks you want taking the expensive route ahead of time.

[Stryker]

Do you even need policy based routing? Like I said, you can make your default route your cheap line and then add static network routes for address ranges you want to go out the expensive interface:

# route add default gw 204.15.230.1 eth1
# route add -net 12.12.0.0 netmask 255.255.0.0 gw 64.34.37.225 eth0

12.12.0.0/16 addresses will take the expensive route while everything else takes the cheap way out. Add as many static network routes as you need. That all assumes that you know which networks you want taking the expensive route ahead of time.

There's too many addresses to do them individually. I just want everything to respond on the same interface it comes in on.

It's just weird because half the time this method works, and sometimes it doesn't, even when I do the same thing.

This is what I have run during bootup now:

sleep 60
ip rule add from 64.34.37.0/27 table 1
ip route add default table 1 via 64.34.37.225
ip rule add from 204.15.230.0/24 table 2
ip route add default table 2 via 204.15.230.1
ip route flush cache

usually on bootup, it doesn't work. I check the rules and routes and they are there. I delete them, run that script manually, and then it starts working. I've added the sleep 60 to the top in case its run before the network starts (its scheduled for boot through cron).

just tested it a few times, and i think its working great. If there's ever an exception (which would be rare) i can add it easily.

Frusterating, I did the same thing yesterday and it wasn't working, and now it is.

[Void Main]

What distro are you running? In Red Hat/Fedora I usually add the routes in their proper place (/etc/sysconfig/network-scripts/route-ethx), where ethx is the interface you want those routes added for, but that is for standard static routes, not sure what would have to be done for policy based routes. Another place you could add the routes is by creating a /sbin/ifup-local script. This gets called after the interface is brought up and the interface name is passed as the first argument. So you would check "$1" in the script and if it equals "eth0" you would add your eth0 routes, eth1 add your eth1 routes. There is good reading on the network scripts and variables in /usr/share/doc/initscripts*/* (at least in Fedora/Red Hat).

Did that link I pointed you for linuxjournal not help at all? It looks to explain how to do exactly what you are trying to accomplish.

[Stryker]

i'm using fedora core 3, and those files don't seem to be there. perhaps i'd need to make them myself. However, this method does seem to be working now and i'm happy with it. The link you gave me did help, I learned I needed to flush the route. I did things slightly different, but I think I have the same results:

ip rule show:

Code: Select all

0:      from all lookup local
32764:  from 204.15.230.0/24 lookup 2
32765:  from 64.34.37.0/27 lookup 1
32766:  from all lookup main
32767:  from all lookup default


ip route show table 1:

Code: Select all

default via 64.34.37.225 dev eth0


ip route show table 2:

Code: Select all

default via 204.15.230.1 dev eth1


[Void Main]

Yeah, those files don't exist unless you create them. The network configuration GUI (system-config-network) will also create them if you edit an interface and add routes. Glad it's working, I might actually do something similar on machines at work as ALL of our systems at work have at least 3 interfaces and could benefit from this. I'll have to do some more research and see if there are some more standard configuration files to put these policy based routes in.