VRRP 85
VRRP Increasing Reliability and Failover with the Virtual Router Redundancy Protocol | |
author | Ayikudy Srikanth, Adnan Adam Onart |
pages | 540 |
publisher | Addison Wesley |
rating | Recommended |
reviewer | Peter H. Schmidt |
ISBN | 0201715007 |
summary | An in-depth review of VRRP that is loaded with helpful and informative details. |
Protocols that allow for automatic failover to a backup router have been around for a while, but they are proprietary, including Digital's IP Standby Protocol (IPSTB) and Cisco's more well known Hot Standby Routing Protocol (HSRP). These protocols have been used successfully for years, but as with all proprietary protocols, they lock users into one vendor. Plus, the test of time has shown there are ways in which they can be improved upon. VRRP has been developed in response.
VRRP lets you set up groups of routers to cover for each other, with each group acting like one virtual router with it's own MAC and IP addresses. If the main router in a group should go down, the others will quickly (in under 3-4s, typically) notice and one of them will be elected to take over. VRRP makes it easy to set up multiple routers with multiple WAN connections and make sure that WAN connectivity won't be lost if a router goes down.
As the open alternative to HSRP, you can count on VRRP being widely supported by the router vendors. Even Cisco is shipping it now. If you design, build or install routed networks of any size, VRRP is something you probably need to learn about.
Read This Book!
Let me just say up front that I think this is a very good book, and worth the read. As the title says, it is all about how to increase the reliability and failover capability of your network. VRRP is its subject, but it is treated with a thoroughness and attention to context I have rarely seen in a protocol text. Perhaps that follows from the fact that reliability and availability are only of concern due to economics; few protocols are developed to meet a business need, so most books on them never need to get beyond defining where they fit in their protocol family. Despite the context material, I found it easy to jump to the low-level technical details, yet was somewhat surprised to find myself actively enjoying the extensive introductory material.
Srikanth and Onart have put a wealth of background into the book. The first chapter treats network availability from a theoretical perspective, but does it so clearly and enjoyably that I read it straight through and felt I had learned some valuable new concepts. It also gave me all the context necessary to easily follow their discussion of the need for, conceptual operation of, and benefits from VRRP in Chapter 2.
Part II, comprising Chapters 3-6, presents the protocol, discussing its messages, state machine and issues with different LAN technologies, firewalls, tunnels and VPNs. I found this a useful complement to the RFC. Here is where I found the details I always look for first when confronting a new protocol: how many messages are there, how many states, what kind of implementation trade offs are going to be necessary? I particularly appreciated the abundance of clear, annotated diagrams in this part of the book, though they aren't confined to these chapters alone.
The chapters of Part III concern themselves with managing VRRP, and what is noteworthy here are the numerous examples of how you can configure and manage realistic scenarios. Juniper and Nortel routers are used in the examples, and you are given step by step instructions on using SNMP, the CLIs and a GUI (HP Openview). If I had to set a customer up with a redundant router configuration tomorrow, I would grab these chapters first thing.
Part IV may be the most useful part of the book for the experienced network engineer. Chapter 10 presents an excellent discussion of the pros and cons of VRRP vs HSRP and IPSBP, and includes some nice summary tables. Chapter 11 discusses the future of VRRP, and answered many of the niggling "How would it handle this scenario?" questions which had popped up as I read how the current VRRP works.
The final section of the book is comprised of 200 pages of appendices. They start with a moderately brief but well done overview of TCP/IP and IP networks. That is followed by the complete VRRP MIB. Then we are given Linux source from http://w3.arobas.net/~jetienne/vrrpd/ and a nice commentary on it. Next is a thorough explanation of the SDL (Specification and Description Language) and flowcharts which were used to define the state machine in Part II. And if this isn't enough rigor for you, you'll be pleased that the following appendix using first-order predicate calculus to specify VRRP yet more clearly. (If you don't know what first-order predicate calculus is, just feel lucky and skip that part...) The final appendix covers UML, the Unified Modeling Language, which also is used in Part II to show how VRRP state transitions occur.
What's Not to Like?
There is very little to object to about this book. If it has a fault, it may be that it is a bit too comprehensive. VRRP is actually a fairly simple protocol, and I write that as someone who has designed and implemented protocol stacks for over 8 years. The level of rigor and detail put into VRRP in this book are worthy of something as hairy as OSPF or BGP4. I found myself getting lost in the notational details of their examples at times, they were so exactingly detailed, but I found that if I just looked at the diagram and skipped to the last paragraph, I'd get what I needed. This book would actually make a pretty good reference book on networking in general, there's so much here!
FAQs
What level of experience is needed to make good use of the information in the book?
This book has all the intro material a novice to networking could want, yet has it all so well organized that it is easy for the advanced reader to find the interesting details.
Who will find it most useful? Is there an existing, canonical book that already covers the same ground?
I think this book would be most useful for enterprise network designers, implementers and operations people, no matter what their current skill level. I couldn't find another book on VRRP, so it's good that the only game in town right now is a good book.
Is the book readable as well as technically accurate? Is the language stilted, or natural? Are examples easy to follow?
The book is very readable - unusually so. As for accuracy, I didn't notice anything amiss, and I used to QA stuff like this. Good use of language, and a ton of excellent examples.
Is the depth appropriate?
This book dives deep, but not without plenty of warning and acclimatization first for people not quite ready for the open ocean.
Are the illustrations appropriate and well executed?
Excellent, clear illustrations.
Do any extras come with the book, like a CD-ROM of additional information or code samples?
You get a full printout of the VRRP MIB, as well as commented source code.
What's missing from the book? Would it benefit from illustrations, a better index, a final chapter on practical applications?
Nothing significant.
You can purchase VRRP from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Patent issues? (Score:2, Interesting)
Re:Patent issues? (Score:1)
It's a shame, because the protocol isn't that bad. (though, certain implementations and their tendency to conduct VRRP wars may be
The whole thing made me look for an alternative. I ended up investigating the Linux-HA [linux-ha.org] project. They didn't really have support for failing over when the box became unreachable from the network (this is a desired behavior with certain shared storage apps and such) so I concocted a plugin called ipfail. All of that has since been included in the recent releases of heartbeat. It's sort of a second-best solution as I think VRRP is really the answer here, but hopefully others will be able to benefit from it.
Re:Patent issues? (Score:2)
Re:Patent issues? (Score:1)
Re:Solution from a problem that doesn't exist (Score:2)
VRRP is very commonly used in the enterprise and it easily and simply saves downtime on a regular basis. Basically, if your enterprise runs two or more layer 3 switches you'd be stupid not to use t. Now think about those networks that use thousands of layer three switches, networks with tens of thousands of routers, virtual or otherwise. In these environments a failure could affect a very large portion of the network. With VRRP, the risk is greatly reduced, though not eliminated.
You've a long wait for IP6 (Score:1)
Re:You've a long wait for IP6 (Score:2)
Anyway, IPv6 ain't that bad. You very quickly become able to memorise the portions of the address relevant to your connectivity and just ignore the rest (or more accurately, drop it:-))
Re:You've a long wait for IP6 (Score:1)
Re:You've a long wait for IP6 (Score:2)
You don't want to have to rely on an application for security when the underlying layers may have been subverted. In fact the best solution is to have security functionality at all levels - keep non-repudiation at a high layer and the rest at each layer lower down.
Wow! (Score:2, Interesting)
I'm excited that Sominex is now available in paperback form.
This is the type of stuff I read on a need-to-know basis.. Ie; I need to implement it, so I get the book. Until then, just knowing what VRRP stands for is enough for most.
Re:Wow! (Score:1)
Re:Wow! (Score:1, Interesting)
where things do get a bit more complex, is in the decision processes that specific implementations feed into setting priorities in VRRP packets. if you say, this VRRP instance has priorty 200, but less 20 if interface X goes down, which makes for a imensely more useful VRRP implementation, you start infringing more patents. nortel have one in europe i think that covers adjusting priorties according to interface states, i think theres a more general one too that bases decision on the presence or lack of specific routes in the routing table.
until these patents expire, vrrp is a closed protocol. but, it's soo simple anyway, that unless you are specifically concerned about interoperating with other boxes on the network, there is usually "another way" that's "just as good".
the most interesting thing about vrrp is that it makes a very clear case of how patents can be abused to the disadvantage of non-corporations.
Router problems (Score:4, Insightful)
In my experience, downtime is caused more by router misconfigurations and not physical problems: peer router broadcasts bad information and takes down a network, admin discovers that the backup route wasn't configured properly at all, admin reboots a set of routers only to find that the TFTP server hosting the OS hasn't been running for 3 months, etc. :)
The protocol seems like it will do nothing to address this. However, physical outages do tend to last longer than misconfiguration outages, so this protocol may help yet.
Re:Router problems (Score:3, Informative)
And honestly, if you're having significant problems due to router misconfiguration, you should really consider replacing your network staff. A few glitches are to be expected, but if they're causing more downtime than your hardware, perhaps they need to study up before they're allowed near your production kit.
Re:Router problems (Score:1)
Re:Router problems (Score:2)
As far as your point on physical outages lasting longer...that's very true. If I can console in via modem or telnet, I can fix a misconfiguration.
Re:Router problems (Score:1)
What is your experience based on? I work in the NOC of a major telecom provider and I would say 80% of outages are related to physical circuit problems or router/card crashes. Configuration issues usually get noticed and solved during the implementation phase of a network.
Re:Router problems (Score:2)
What is your experience based on?
Working with ISPs. Most of the time any outages are explained to us as a router misconfigurations or some weird event happening that caused some weird problem. At this level the ISP peers with a lot of other third party ISPs so there's naturally a smaller degree of coordination. I couldn't speak for large internal organizational networks.
Who knows, maybe the ISPs have dozens of hardware failures that they recover from without missing a beat.
not really all that practical for ISP of any size (Score:2)
Re:Exactly how are routers going to run this ??? (Score:2)
I don't think any of my clients would ever think of not using it - sometimes for firewall resiliency, sometimes to help with web farm clustering, but for large scale enterprise organisations it is invaluable.
Re:Just gateway redundancy (Score:1)
Reliability of TCP (Score:1)
Huh? It seems I must have missed something important...
As far as I knew the TCP/IP architecture was designed to provide a very failsafe and reliable network which should be able to keep running if half of it would be blasted away by a nuclear attack.
But it isn't? Damn... Hey, let's not tell it to the American government!
Re:Reliability of TCP (Score:2, Interesting)
We're dealing with the IP level here.
The internet is supposed to be reasonably intelligent about routing traffic around broken hardware, but the hardware that performs this task is not designed to be redundant/resilliant.
i.e. if your core router breaks, your core breaks.
VRRP is all about getting two routers/firewalls to act as a single unit, intelligently failing over as required.
I've been using VRRP for about 5 years now on Nokia IPSO and Linux boxes. I have to say it's a great solution. It really does mean that all your problems are config errors. Hardware failure is not an issue.
Re:Reliability of TCP (Score:1)
I've also setup many Nokia IPSO systems and always felt good they have been ahead of most people by natively supporting VRRP. Monitored circuit is also pretty much a requirement too. The problem with hot-standby systems though is that there is always one system sitting idle. At first it's fine, but after a few years you think "Why do I have two devices (router, firewall, etc.) and only one is ever active?". And no, running something like OSPF or a load balancer to rotate between multiple VRRP addresses just adds nothing but uncessary complexity to a design. Nokia is now starting to push their clustering in IPSO 3.6 as a replacement for VRRP. For those of us who have used Nokia IPSO systems, VRRP is certainly nothing new, but by the time the rest of the industry finally picks up on it the we'll probably have dropped it in favor of clustering.
What amazes me most though is how people manage to write 560 page books on a topic that can be discussed in a few pages.
Re:Reliability of TCP (Score:2, Informative)
Side note: When your service providor says that they have "99.999%" uptime to the Internet, ask them if their up-stream providors ever have routing/router problems... Since "Five Nines" a year equates to "five minutes" of downtime per year, a single BGP route update is five minutes. In the real world, if an ISP is up >99.9% of the time (99.9% == 8.8 hours) using a simplistic monitoring (pinging their outside routers, pinging the firewall I am behind, etc) then I am happy.
good review (Score:3, Funny)
hmm (Score:3, Funny)
This is awful!
How can I look down my noses at the others in my office with the air of superiority that comes with understanding more of the geeky technicalities of our computer system than they do?
If they keep writing technical books that people can understand, we'll all be relegated to memorizing the stats on Pokemon playing cards!
Burn this book. It's a witch I tells ya.
Not Open enough for OpenBSD (Score:3, Interesting)
I have come to see the merits of this position (dhartmei@ makes a good case, btw
Why does Cisco do this? It may be that they are pissy because HSRP wasn't accepted as the standard. Or they might be looking for protection as they might be afraid other people could have patent claims against stuff that might underlie VRRP - and thus these encumberances could allow them to enact a legal war of attrition as each side has competiting patent claims that would ensure deadlock.
ostiguy
HSRP can be a pain in the ass. (Score:2)
For example, the Nortel Passport 8000 will clobber your HSRP group if in the unfortunate instance two HSRP routers send hello packets at the same time. As far as the switch goes, it sees a packet from the same source address in two different ports at the same time...then it shuts off the ports for a short bit, and when it all comes back, both routers think they're active because a two-way dialoge can't happen due to switch blocking. The only known remedy that I've heard of is to reset the switch and that disrupts everyone else's communications.
I got the whole scoop on this problem dealing with a major east coast University this weekend. According to Nortel, this is not a bug, so VRRP is on the docket to be implemented.
HSRP is great...but unless you're running an all-Cisco shop, it can be a pain in the ass. This is why a standards-based protocol was needed.
Re:HSRP can be a pain in the ass. (Score:2)
I don't know if you've looked at the packet formatting and design of HSRP versus VRRP, but they're pretty much the same. They both implement a multicast group to which both sides broadcast, sourcing from the configured interface IP, at regular intervals. They both use a half-duplex configuration mode a-la PPP's LCP negotiation, at which point they elect the virtual MAC and decide who's going to own it. After that, they make an ARP broadcast so that all the hosts on the segment know where their virtual gateway is. If the hellos ever fail to arrive in a timely fashion to the secondary (or tertiary or whatever, priority is configurable for both protocols), the secondary sends the same ARP broadcast to updated the L2 forwarding database and the hosts' ARP tables. Depending on how you set it up, when connectivity to the primary is restored, it either takes back ownership of the virtual gateway, or leaves it on the secondary.
As for standards-based, HSRP is described in detail in RFC 2281.
Nokia IPSO comes with VRRP standard (Score:2)
Re:Nokia IPSO comes with VRRP standard (Score:2)
ostiguy
Re:Not Open enough for OpenBSD (Score:1)
Cisco just said - if you have patents on VRRP - do not sue us and we will not sue you. Nothing more, nothing less.
OpenBSD guys are just overreacted IMHO. FreeBSD and Linux guys do not care...
No Open Source implementation because of Patent? (Score:3, Interesting)
I am not aware of any open source project that has ships VRRP. The IETF has received more information from Cisco about their Intellectual Possession [ietf.org] in regards to VRRP.
Implementations for Linux and FreeBSD (Score:2, Informative)
For FreeBSD [bsdshell.net]
For Linux [linuxvirtualserver.org]
Publisher's site/sample chapter. (Score:3, Interesting)
Enjoy
[ More Pages Like This ] (Score:5, Informative)
VRRP Internet draft
VRRP for IPv6
VRRP RFC
Email the chair of VRRP
Mailing list archive
Nokia (Score:1)
Sure, many do. (Score:1, Offtopic)
Lucent - Cajun Series
HP - Pro Curve
Foundry
Alcatel
IBM
etc....
VRRP OpenSource code (Score:4, Interesting)
Wackamole (Score:2, Informative)
Reliance on Secondary Routing Protos... (Score:3, Interesting)
One problem with VRRP v2 as it stands today. Imagine a case where you have two parallel routers and are running VRRP. If you experience an interface failure on your primary router. Ok, that interface fails over to the secondary unit. Since you only experienced an interface failure (let's suppose this is a pair of edge routers), say on the outside. Because the inside i/f of the router is still up, you need a secondary routing protocol to direct the traffic to the secondary router - introducing an asymmetric routing condition. This is easily done with OSPF.
Consider the case, however, that we're no longer talking about routers, but instead firewalls. This condition can wreak havoc with your firewalls state tracking mechanism if your firewall's connection state tracking mechanism is either not shared with the redundant unit, or your connection is fast enough that reply packets arrive before connection data is sync'd.
Enter extensions to VRRP like VRRP Monitored Circuits (aka VRRPmc), from Nokia. If you're running Nokia firewalls (which run Check Point for those who don't know), you're probably using VRRPmc.
When you configure VRRPmc, you monitor the other interfaces in use for VRRP. If one of those other interfaces goes down, you decrement your VRRP priority value by a pre-defined delta value, which if you've calculated correctly, will cause the primary unit to begin advertising VRRP priorities that are lower than what the secondary unit is advertising, thereby causing the virtual ips/macs to shoot over to the secondary unit, rather than just the i/f that failed. On the wire, it still looks like good old VRRP. I'd like to see either the monitored circuits method, or something similar implemented in the mainstream VRRP protocol.
Re:Reliance on Secondary Routing Protos... (Score:1, Informative)
Re:Reliance on Secondary Routing Protos... (Score:1, Informative)
Firewall Load Balancing
Alteon switches (in a sandwiched pair) maintain persistency across firewalls while load balancing and health checking all the way though clean and dirty sides to ensure redundancy.
Re:Reliance on Secondary Routing Protos... (Score:2)
The Nortel solution uses something called 'critical interface' which is not necessarily an ethernet interface. You associate a 'critical interface' with the interface you are running VRRP on. When that 'critical interface' goes down, it causes VRRP to give up the master status on that device, thereby yielding to the other VRRP peer...
I think this is what you mean.
"locked" in to one vendor (Score:1)
For example...the mtu size for a cisco router by default is 1500. On a nortel box, it's something like 1600 or so (don't remember off the top of my head). Anyhow, this isn't a big deal, until you start getting lots of remote sites connected. Then, it becomes a huge issue.
Anyhow, for my money, corporations should use one vendor for all their needs. That way, you know that the routers/switches, whathaveyou will work and play well with each other.
Re:"locked" in to one vendor (Score:1)
VRRP isn't the end-all-be-all (Score:3, Informative)
Also, there's the issue of the L2 packets being broadcast, so that when the switch you're connected to stops forwarding unicast (oh, but broadcast and multicast still work just fine, thank you very much), VRRP is pretty much useless. It'll never realize that something's gone horribly wrong, and that failing over is necessary.
I guess my point is that most of the "automated failover" solutions out there either are or have been pretty much worthless given the failures I've seen, and that VRRP, for all it's good points, only covers about ten percent of failure situations. For straight up gateway reachability, it does just fine (in fact, it's a nearly complete rip of Cisco's HSRP, altho I'm not exactly sure of the timeline for each protocol), and in fact it's a superior solution in that regard, but for anything else other than L3 gateway services where all you're doing is plain-vanilla IP routing, it's pretty lousy.
What I'd like to see is a unicast-based, fully-configurable hot-standby solution. Something where you're forced to enter the IP of the other partners in the redundancy group. Simply sticking to broadcast- or multicast-based solutions isn't going to cut it in a fully switched environment. Granted, the above is going to require a bit more configurationbut come on, it would add what, one, maybe two lines to each configured group? Hell, my environment has two hundred configured interfaces like this, and I'd put up w/ the extra work if it would have saved me from some of the failures I've had.
Big Problem with VRRP That No One Ever Mentions (Score:2)
The biggest problem I've run into with VRRP -- no one here has mentioned it, and neither does the author of the book -- is that it insists on using MULTICAST MAC addresses for the virtual routers.
Why's that so bad? Because Cisco routers refuse to accept Multicast MAC addresses as responses to their ARP requests.
That means: VRRP no worky, if you're connecting to a Cisco box.
If you happen to admin the Cisco routers, fine, you can change the settings, etc. But what if the next router upstream is run by your ISP? And what if the ISP won't make the change?
There are better solutions; heartbeat (www.linux-ha.org) is usually sufficient for routers. Now if VRRPv3 comes out and they've made the multicast MAC thing optional, I will be happy to change my mind.
great review; simple protocol (Score:2)
While it is simple, the way it works can make troubleshooting complicated. Specifically, it 'breaks' the traditional way IP and Ethernet addresses relate to each other. By responding to ARP's for more than 1 IP address, and by receiving and routing traffic destined for more than 1 MAC address, it breaks certain assumptions you would make when troubleshooting a problem at the ethernet level.
This could be very misleading behavior, if you didn't know to expect it...Not like a routing protocol where you can see the effect it has on routing information.