Before we get into the layer 3 troubleshooting methods, we first need to make sure we have a basic understanding of how routers and multilayer switches route traffic.  Three tables are used: the routing table, ARP table, and CEF mappings.

The routing table pairs network prefixes with the router’s preferred next hop address or interface.  Packets are routed based on the output of the routing table by first matching the longest prefix and then using other IGP-specific metrics.  The show ip route command displays the contents of the routing table.

After the router has determined what the next-hop address is, the router then needs to translate that into a layer 2 MAC address.  The ARP table is exactly what this is for.  The show ip arp command will display the current ARP pairings.

Lastly, CEF is used in layer 3 switches to optimize routing and layer 2 headers.  To view the CEF entries, use the show ip cef command.

Troubleshooting Any Routing Protocol

Regardless of what routing protocols are in use, there are some common troubleshooting steps that can be applied.  First, try to ping the destination to determine reachability.  Next, look at the routing table to make sure a route to the destination exists.  Finally, run a traceroute from the source towards the destination to see where the last reachable hop is.

For further digging, the show ip protocols command gives some very helpful information on the current routing protocols in use (like timers, AS numbers, etc.).

Routing Protocol Troubleshooting Methodology

There are three key questions that can be extremely helpful when troubleshooting a routing issue – regardless if you are running EIGRP, OSPF, or BGP.

1. Is the route being advertised properly?

2. Is the route being received?

3. Is there a more desirable route being used (longer prefix or lower administrative distance)?

Now let’s dissect each of these for the major routing protocols one at a time.
 

EIGRP

First, verify connectivity to the remote networks using pings and by taking a look at the local routing table.

As a reminder, EIGRP stores its information in three different tables: the EIGRP interface table, neighbor table, and topology table.

EIGRP Interface Table
The EIGRP interface table displays interfaces participating in the local EIGRP processes.  Use the show ip eigrp interface command to display its contents.

EIGRP Neighbor Table
The EIGRP neighbor table contains a list of discovered EIGRP neighbors.  Use the show ip eigrp neighbors command to display its contents.

EIGRP Topology Table
The topology table contains a complete list of EIGRP-learned routes. Use the show ip eigrp topology command to display its contents.

Is the EIGRP route being advertised properly?

Remember those three troubleshooting questions listed above?  Let’s start with the first one – is the route being advertised properly?

The first step is to identify the router that is connected to the destination subnet as it should be advertising the route out.  There are two simple ways to check if that router is advertising the routes properly.

First, do a show run | section eigrp.  This will display the running EIGRP configuration, including what networks are being advertised with the network statements.

Another option is to do a show ip protocol.  The nice thing about this command is that it displays the EIGRP network statements.  Remember, EIGRP only advertises subnets of interfaces that match an EIGRP network statement.

Is the EIGRP route being received?

Routers must be EIGRP neighbors for the routing information to be shared.  To check this, issue a show ip eigrp neighbors on the two routers exchanging hellos.  You should see the neighbor listed on each device.

You can also perform a debug ip eigrp packets to make sure hellos are being sent out from each router.

If all of that looks good, look at the EIGRP running configuration and make sure the AS numbers match, the timers are close, and that any authentication configurations are the same.

Next, issue a show ip eigrp interface to make sure the interfaces you expect are participating in the EIGRP process.  Lastly, route maps or distribution lists could be blocking routing traffic.  Do a show ip protocols to display any distribute lists.

Is there a more desirable route being used?

It’s possible that EIGRP knows about the route, but it is not being used in the routing table.  If a more desirable path is known, that will be used instead.   Compare the EIGRP topology table to the local routing table.

OSPF

These steps for troubleshooting OSPF are very similar to EIGRP.  First, verify that there is a problem using pings and by taking a look at the routing table.

OSPF stores its information in three different tables: the OSPF interface table, neighbor table, and link-state database.

OSPF Interface TableThe OSPF interface table displays interfaces participating in the local OSPF processes.  Use the show ip ospf interface command to display its contents.

OSPF Neighbor TableThe neighbor table contains a list of discovered OSPF neighbors.  Use the show ip ospf neighbors command to display its contents.

OSPF Link State Database
The link state database contains the received LSAs. Use the show ip ospf database command to display its contents.

Is the OSPF route being advertised properly?

The first step is to identify the router that is connected to the destination subnet as it should be advertising the route out.  There are two simple ways to check if that router is advertising the routes properly. 

First, do a show run | section ospf.  This will display the running OSPF configuration, including what networks are being advertised with the network statements.  Another option is to do a show ip protocol. 

Remember, OSPF only advertises subnets of interfaces that match an OSPF network statement.

Is the OSPF route being received?

Routers must be OSPF neighbors for the routing information to be shared.  To check this, issue a show ip ospf neighbors on the two routers.  You should see the neighbor listed on each device.

You can also perform a debug ip ospf adj to show any issues that would prevent the routers from forming an adjacency.

OSPF is more particular about matching protocol variables than EIGRP.  OSPF requires that all of the following parameters match between devices:

• Bidirectional communication
• AS number
• Timers
• Common area type
• Common subnet prefix
• Authentication

The OSPF protocol values can be seen using the show ip ospf interfaces command.

Lastly, route maps or distribution lists could be blocking routing traffic.  Do a show ip protocols to display any distribute lists.

Is there a more desirable route being used?

It’s possible that OSPF knows about the route, but it is not being used in the routing table.  If a more desirable path is known, that will be used instead.   Compare the OSPF topology table to the local routing table.  Take the time to check each hop along the expected  path and look at the routing tables on each router.

BGP

BGP stores its information in two tables: the BGP neighbor table and the BGP table.

BGP Neighbor Table
The neighbor table contains a list of known BGP neighbors.  Use the show ip bgp neighbors command to display its contents.

BGP Table
This table contains all the received BGP prefixes as well as their associated attributes lists.  Perhaps most importantly, it also shows the BGP best path to each destination. Use the show ip bgp command to display its contents.

Are the BGP routers neighbors?

BGP neighbors must be administratively assigned on each router running BGP.  If the routers are not neighbors, BGP routing and network information will not be passed between them.  Start by doing a show ip bgp neighbors.  If the expected BGP peers do not show up in the output, make sure they have L3 connectivity using a simple ping test.  If you need to investigate further, a debug ip bgp updates should show the BGP hellos and advertisements.

Remember that BGP requires bidirectional communication as well as matching AS numbers and authentication.  The show run or show ip bgp command will display that information.

Also, consider that route maps or distribution lists could be blocking routing traffic.  Do a show ip protocols to display any distribute lists.

Is the BGP route being advertised?

As with the other routing protocols, make sure that the router connected to the destination subnet is advertising the route out.  There are two simple ways to check if that router is advertising the routes properly.

Perform a show run | section bgp to look at the neighbor statements.  You should also keep in mind that BGP will only advertise routes when (1) they are defined using neighbor statements and (2) the router knows about the route from another source.

Route Redistribution

Route redistribution can be a tricky situation to troubleshoot, but understanding the following concepts should be helpful.

1. Redistributed routes require an existing entry in the routing table.  If the redistributing router does not have a routing table entry for the route being redistributed, it will not work.  Seems simple, but it should checked right away.

2. Routing loops are a common problem with multi-router routing redistribution.  Use a single router to perform the redistribution if possible.

3. Understand that redistributed routes lose their native metric information.  When redistributing into EIGRP, a default metric MUST be set or no route will be imported.  When redistributing into OSPF, all routes will be imported as classful unless the subnets keyword is appended to the end of the redistribution statement.