Real+World+OSPF

Here's a quick example of how network graphs can help you visualize what's going on in your network (despite the page name, this has only a little to do with OSPF):

For a while now, we've had a problematic T1 circuit connecting 2 of our Points-of-Presence (POPs). Due to the nature of how the circuit is built through the Telco (long story short - it rides numerous DS3's and the Telco doesn't have the circuit properly documented), fixing the underlying problem is a feat unto itself.

... here's a small snippet of the important parts of this network segment (all links below are T1)::

The serial link in red is a problematic T1 (with HDLC encapsulation). When the T1 periodically falls over, OSPF denotes the dead-timer expiring and the usual routes that pass directly to Carlisle are removed and replaced with the route through Blountville to Carlisle.

Network graphs further indicate what is happening (these graphs are all from the perspective of the Ijamsville, MD router)



As you can see, in the highlighted (in yellow) periods, the "T1 to CRLS" circuit falls over, OSPF updates thus removing the routes using the T1 to Carlisle, and bandwidth utilization on the T1 to Blountville increases. Eventually the circuit comes back up on it's own.

Router logs are sent to a //syslog// //server// - that server denotes the following as happening in this timeframe:

Sun Feb 21 17:25:44: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial2/0, changed state to **up** Sun Feb 21 17:25:54: %OSPF-5-ADJCHG: Process 10, Nbr 66.xx.xx.0 on Serial2/0 from **LOADING** to **FULL**, Loading Done

Mon Feb 22 18:33:32: %OSPF-5-ADJCHG: Process 10, Nbr 66.xx.xx.0 on Serial2/0 from **FULL** to **DOWN**, Neighbor Down: Dead timer expired

Tue Feb 23 06:17:31: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial2/0, changed state to **up** Tue Feb 23 06:17:40: %OSPF-5-ADJCHG: Process 10, Nbr 66.xx.xx.0 on Serial2/0 from **LOADING** to **FULL**, Loading Done

Wed Feb 24 16:23:07: %OSPF-5-ADJCHG: Process 10, Nbr 66.xx.xx.0 on Serial2/0 from **FULL** to **DOWN**, Neighbor Down: Dead timer expired Wed Feb 24 16:23:32: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial2/0, changed state to **down**

Sun Feb 28 09:34:55: %OSPF-5-ADJCHG: Process 10, Nbr 66.xx.xx.0 on Serial2/0 from **FULL** to **DOWN**, Neighbor Down: Dead timer expired Sun Feb 28 09:35:14: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial2/0, changed state to **down**

Sun Feb 28 19:18:25: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial2/0, changed state to **up** Sun Feb 28 19:18:34: %OSPF-5-ADJCHG: Process 10, Nbr 66.xx.xx.0 on Serial2/0 from **LOADING** to **FULL**, Loading Done

Sun Feb 28 20:55:15: %OSPF-5-ADJCHG: Process 10, Nbr 66.xx.xx.0 on Serial2/0 from **FULL** to **DOWN**, Neighbor Down: Dead timer expired Sun Feb 28 20:55:31: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial2/0, changed state to **down**

Having a //syslog server// in a multiple router environment is a huge bonus - instead of perusing logs on each individual router, all logs are stored in a central location. Another big benefit is that sometimes you'll focus on routerA and routerB, while routerC is doing something to cause problems. The all-on-one storage location gives you an easy way to look at events within in a specified timeframe without having to log into each router.

Another //extremely// nice thing to have in a multiple router environment (at least IMHO) is some form of a graphing system (like the open source programs NetMrg and [|Cacti] ) - something that allows you to look back at the last few weeks for a historical review of what your network usually looks like - is a big plus.