World IPv6 Launch (June 6th 2012) is
fast approaching, so I thought I'd share some details about IPv6 deployment
at the University of Pennsylvania and what we've recently done to prepare
for this event.
A quick history
Penn runs a regional network called MAGPI,
which connects Research & Education (R&E) institutions in our area
(eastern Pennsylvania, New Jersey, and Delaware) to national R&E
backbone networks like Internet2. We first
deployed IPv6 in the MAGPI network in mid 2002 and soon after,
established an external peering with Internet2. At that time, a small
number of engineers in the networking department (including myself)
typically had our computers directly wired into MAGPI infrastructure
to get IPv6 connectivity at desktops and test servers.
IPv6 was introduced more gradually into the Penn campus network
infrastructure, starting in 2005. Initially it was enabled only at the
border and core routers, and extended out to only selected IT
departmental subnets. In September 2005, Penn hosted the Fall
Internet2 member meeting
in Philadelphia, where we operated the conference
at the Wyndham Franklin Plaza hotel - this network was fully IPv6
enabled, including support for IPv6 multicast routing. (Incidentally,
we are hosting the Fall 2012 Internet2
meeting this October
again, so I hope to see some of you there.)
Over the course of the years since, we've been gradually extending
IPv6 network connectivity to the rest of the campus, and turning up
IPv6 enabled application services where feasible. Needless to say, it
is still early days in IPv6 deployment and a huge amount of work
remains to be done.
Unlike other IT services at Penn, many of which are highly
decentralized, the campus network is mostly run by the central IT
organization - this gave us the ability, when needed, to roll out IPv6
to large portions of the network fairly rapidly. Due to many competing
priorities and projects, we have mostly not taken advantage of this
ability, until quite recently. IPv6 had been deployed on departmental
subnets only where it had explicitly been asked for. One of the more
interesting cases was the Annenberg School for Communication - they
approached the central IT group a few years ago with a need for IPv6
in order to facilitate some collaboration with partners in China who
had asked if they'd be able to conduct video conferencing over
IPv6. This was the first time we encountered direct external pressure
to deploy IPv6. I'm sure it won't be the last.
The one subdivision within the university that does run their own
network infrastructure, the School of Engineering & Applied
Science, has been an early adopter, and
has been running IPv6 in their part of the network since 2007.
In the summer of 2011, we took advantage of the increased interest
generated by last year's World IPv6
Day event to extend the deployment of
IPv6 to most of the rest of the campus wired network. The one area
that was significantly lagging was the wireless network. This was a
bit more challenging because of known bugs in our wireless controller
vendor's gear (Aruba Networks) which necessitated a code upgrade. That
code upgrade did not happen until earlier this year, so we're still in
the midst of IPv6 deployment on wireless. As of this writing, 70
wireless subnets (out of roughly 200) have IPv6 available, and we
should have the entire wireless network done sometime later this
For the more technically inclined, we run Integrated
IS-IS as our interior routing
protocol for IPv6, whereas we continue to run OSPF for IPv4. At the
time when we were initially testing IPv6, that was clearly the best
choice since OSPF version 3 (the new version of OSPF that supports
IPv6) was still in a relatively fledgling state of implementation
maturity. Also confining IPv6 to a separate routing protocol seemed
like a good additional safety measure. We run a single flat Level-2
area for the entire campus. For exterior routing, we have separate BGP
peerings over IPv6 transport established with our external peers that
carry IPv6 routes only. Our initial deployment used a provider
allocated /48 IPv6 block delegated to us by MAGPI. In 2008, we
obtained a Provider Independent ("portable") /32 sized IPv6 address
block (2607:F470::/32) from the regional registry
ARIN, and have mostly renumbered
Currently, Penn's only connection to the IPv6 Internet is via MAGPI
and Internet2. But we're planning to turn up IPv6 peering on our
direct commercial ISP links (Level3 and Cogent) in the very near
future. At least one of them might happen before World IPv6 Launch.
IPv6 enabled servers use statically configured addresses. Clients on
campus almost exclusively use stateless address autoconfiguration
(including the privacy/temporary address extensions). DHCPv6 has not
been an option for us until recently, since we're a 40% Mac campus,
and Apple didn't support DHCPv6 until Mac OS X version 10.7 (late
summer 2011). We are developing plans for a possible DHCPv6 service
in the future, which I'll elaborate on at a later time.
Penn's authoritative DNS service has been IPv6 enabled for many
years. The campus DNS resolvers also support DNS queries over IPv6 but
since we don't yet run DHCPv6, we don't have a convenient way to hand
out their IPv6 addresses. Our homegrown DNS content management system
has supported the ability to create AAAA and IPv6 PTR records for a
long time also.
A number of departmental web servers, including the School of
Engineering & Applied Science, are IPv6 enabled. The Penn central
jabber server, jabber.upenn.edu, was one of our earlier IPv6 equipped
services, and actually sees a high proportion of IPv6 activity. Work
is proceeding on many other services.
Some of the most challenging services have been those where components of the service have been outsourced to commercial third parties. The central Penn webserver, www.upenn.edu is located on the Akamai content delivery network, and Akamai has been slow to deploy IPv6. We successfully worked with Akamai to put the website on IPv6 for last year's world IPv6 day (June 8th 2011), but they were not then prepared to offer it on an ongoing production basis. In April 2012, Akamai finally announced production IPv6 support. As of May 9th, the Penn website is now available over IPv6, hopefully permanently this time.
Akamai uses DNS resolver client addresses to direct users to content servers geographically close to them (although a few other factors including load are also considered by the server selection algorithm). I collected some data with the help of colleagues about where the www.upenn.edu AAAA record resolves to from various locations. Since we host a cluster of IPv6-enabled Akamai content servers on our campus network, most of the time, on-campus users of www.upenn.edu will be directed to these local servers.
One issue we overlooked, is that there is a version of the main Penn
website optimized for small form-factor mobile devices ("m.upenn.edu")
which is not on the Akamai CDN, and run by another unit within the IT
organization that has not yet deployed IPv6. So, more work remains to
get the Penn web presence completely IPv6 ready.
The other challenging service is central e-mail. Penn uses Message Labs (now Symantec Cloud) to scan e-mail for viruses and spam scoring. As a result both inbound and outbound e-mail has to go through Symantec Cloud's servers. We've inquired about IPv6 support for a number of years, but even today, they appear to have no plans to support it. Our latest communication from them (early May 2012) indicates that they have no plans for any IPv6 support for FY13 (their fiscal year starts in April), and that this may change as priorities shift. At some point, we too might be compelled to shift our priorities and end our relationship with Message Labs, and either seek another provider (does Google/Postini do IPv6 yet?) or bring back virus & spam filtering in-house.
For a comparative view of externally visible IPv6 enabled application services deployed at various US universities and other organizations, Mark Prior's IPv6 survey website is a good resource. Of the five services measured there (Web, DNS, Mail, NTP, and Jabber), Penn gets a green box for four - Mail is the missing one because of Symantec Cloud.
From time to time, we've worked with Penn researchers and outside companies on IPv6 related projects. In the fall of 2009, we worked with Alain Durand (then at Comcast) and Roch Guerin (Penn engineering school faculty) on a small trial deployment of Dual Stack Lite; see RFC 6333 for details of this protocol - this was mostly to help Comcast out. It's unlikely that Penn will deploy DSLite in our own production network. We've also worked with Roch and Comcast on an ongoing IPv6 adoption measurement project. Details of this project are available at: http://mnlab-ipv6.seas.upenn.edu/
Facilitating Regional Connectivity
As mentioned earlier, Penn enables IPv6 connectivity for regional
institutions via the MAGPI GigaPoP and Internet2. Currently, we
provide IPv6 connectivity to the following institutions: Princeton
University, New Jersey Edge (the state education network for NJ),
Lafayette College, and Rutgers University. Of them, Princeton came up
first in 2005.
Looking at some recent data, IPv6 traffic traversing the campus border
is roughly 3% of the total inbound and about 1% of the total outbound
traffic. Internal traffic is probably a slightly higher
percentage. We're just starting to deploy better measurement
infrastructure for IPv6, so we'll have more comprehensive data in the
future. But I'll be writing another article sharing what we have so