Network Diagram
Note: Abilene has since had their allocation increased to a /32, giving them
a 16-bit NLA, but as of of this writing they have not expanded into
these bits yet.
The 8-bits (between /40 and /48) are structured as follows:
It would be nice to subdivide the 6-bit NLA2 field into multiple ranges
based on the access router location that is connecting the customers
numbered in that NLA2 subrange. That way, we might be able to do some
route aggregation from the gigapop access routers into the gigapop core
network. With only 6 bits (and possibly 8 if we expand into the 2 reserved
bits), it is probably not worth the extra effort though. Who cares if we
have to carry 256 /48 routes inside MAGPI.
An entirely different allocation strategy would be to employ Marc
Blanchet's method of bit assignment (specifically the centermost
approach). This would allow us to postpone the decision of where to
partition the address space if we weren't sure about the eventual
size of customer allocations or the gigapop's own allocation. This
scheme complicates the assignment procedure and throws us out of
sync with the rest of the Abilene connectors, so we do not employ it.
Note: using the 11 prefix means that we are using 1/4 of the address
space for the point2point links! This seems a little bit of a waste,
but it makes decoding the v6 addresses easier and it is not as wasteful
as it might seem really. The gigapop is most likely going to be composed
of numerous core/access routers and many links interconnecting them.
There won't be too many internal subnets hanging off the routers.
Note: /64 subnets connecting customers to the magpi access routers are
numbered out of the MAGPI /40 allocation.
Example: Point-to-Point link between phl-01 and phl-02 is:
Use CoreID = 0x3f to number p2p links. Then we have 8 bits of the
SubnetID field to allocate networks out of. We can't implement the
visually simple scheme of 6+6 concatenated bits for the 2 router
coreids because that won't fit in 8 bits. So we maintain a table of
links, which can have 2^8 = 256 links, interconnecting pairs of
routers. Theoretically a full mesh of p2p links between 64 routers is
2016 links, but this is an extremely unlikely scenario. The routers
are more likely to by sparsely linked (physically), so a relatively
small table is perfectly adequate.
We'll go with our original approach for now.
eg. the 3rd subnet hanging off core router 5 is:
2001:0468:1800:0503::/64
eg. Loopback of phl-03: 2001:0468:1800:0300::1/128
Additional loopback interfaces can be assigned by incrementing the InterfaceID.
Multi-protocol BGP is used for inter-domain peering with Abilene. At
the current time, MAGPI's sole IPv6 peer is Abilene, and this peering
point is located at one border router. Hence, in the interests of
simplicity, this border router injects a static default (::/0) into
RIPng to other internal routers.
Additionally, an IPv6 Internal-BGP mesh has been configured between all
the v6 enabled MAGPI routers to propagate external routes to them. This
will become important as MAGPI begins to peer with other entities (eg.
the numerous campuses) at various border routers.
Abilene Notes
Abilene's 13-bit NLA is structured as follows:
+--------------------------+
| NLA1 | R | NLA2 |
+--------------------------+
0 5 7 12
/35 /40 /48
where
NLA1 (5 bits) identifies the 6POP (the Site ID) and
R (2 bits) (Reserved)
NLA2 (6 bits) is assigned by the entity controlling NLA1.
MAGPI has been assigned the 6POP SiteID 0x18, so it's address block
is: 2001:0468:1800::/40
GigaPoP /40 Allocation
2001:0468:1800::/40
The GigaPOP will make /48 sized allocations to it's connected
institutions. At least initially, MAGPI will follow Abilene's general
recommendations about allocation strategies.
R: 2-bit Reserved field for future expansion.
NLA2: 6-bit NLA2 field, out of which subscriber allocations are made.
2 6 (#bits)
+---+---------+
| R | NLA2 | (field name)
+--------------
0 2 8 (bit boundary)
/40 /48
With the 6-bit NLA2 field, we can number 2**6 = 64 customers. The NLA2
field of 000000 is allocated to the GigaPoP itself. The individual
campuses are then numbered sequentially (or with some gaps to allow
possible future expansion of a campus to two consecutive /48's). The NLA2
field of 111111 (0x3f) is reserved for Point-to-Point connections between
the GigaPoP and it's customers.
R NLA2 (aka CampusId)
-- -------------------
00 xxxxxx
00 000000 = 0x00 = 2001:0468:1800::/48 = Campus 00 MAGPI Internal
00 000001 = 0x01 = 2001:0468:1801::/48 = Campus 01 reserved
00 000010 = 0x02 = 2001:0468:1802::/48 = Campus 02 University of Pennsylvania
00 000011 = 0x03 = 2001:0468:1803::/48 = Campus 03 reserved
00 000100 = 0x04 = 2001:0468:1804::/48 = Campus 04 Princeton University
00 000101 = 0x05 = 2001:0468:1805::/48 = Campus 05 reserved
00 000110 = 0x06 = 2001:0468:1806::/48 = Campus 06 Temple University
00 000111 = 0x07 = 2001:0468:1807::/48 = Campus 07 reserved
00 001000 = 0x08 = 2001:0468:1808::/48 = Campus 08 University of Delaware
00 001001 = 0x09 = 2001:0468:1809::/48 = Campus 09 reserved
00 001010 = 0x0a = 2001:0468:180a::/48 = Campus 10 unassigned
00 001011 = 0x0b = 2001:0468:180b::/48 = Campus 11 unassigned
00 001100 = 0x0c = 2001:0468:180c::/48 = Campus 12 unassigned
...
00 111111 = 0x3f = 2001:0468:183f::/48 = Point2Point connections
Point-to-Points (NLA2 = 0x3f):
Point-to-Point connections between MAGPI and it's subscribers are defined
by an NLA2 field of 0x3f and a SLA composed of concatenated 8-bits
specifying the source and destination site. By convention, the source
site is the site with lowest numbered CampusID (ie. NLA2 field), which
is usually always MAGPI. Format:
2001:0468:183f:ssdd::/64
where ss = NLA2 of the Source Site
and dd = NLA2 of the Destination Site
So the point-to-point subnet between MAGPI and Princeton would be:
2001:0468:183f:0002::/64 (src=00=magpi, dst=02=princeton)
A possible improvement:
GigaPoP Internal /48 Allocation
2001:0468:1800::/48
Now we describe how the GigaPoP's own /48 network is used for it's
internal infrastructure.
2001:0468:1800:xxxx:/48
xxxx: MAGPI's SLA field (Site Level Aggregator), 16 bits
The 16-bit SLA field (xxxx in the above) which sits between /40
and /48 is structured as follows:
2 6 8 (#bits)
+---+---------+---------------+
| R | CoreId | SubnetID | (field name)
+-----------------------------+
0 2 8 15 (bit boundary)
Each MAGPI router is assigned a coreid (ie. core router id) of 6 bits.
With 6 bits we can number up to 64 routers.
CoreID Router
0x00 Reserved
0x01 phl-01
0x02 phl-02
0x03 phl-03
0x04 phl-04
0x05 phl-05
0x06 phl-06
...
0x3f reserved (for now)
Point-to-Point Router Links
For point-to-point links between routers, we assign binary 11 to the
reserved bits (R) at the beginning of the SLA. CoreID is the coreid
of the 1st router. SubnetID is redefined as 00 + CoreID of the 2nd router.
Format:
2 6 8 (#bits)
+----+---------+---------------+
| 11 | CoreId1 | 00 CoreID2 | (field name)
+------------------------------+
0 2 8 15 (bit boundary)
So point-to-point links have a siteID of 11xxxxxxxxxxxxxx
ie. they start with c (1100), d (1101), e (1110), f(1111) at the 13th
hex digit.
11 + CoreID1 00 + CoreID2
11 + 000001 00 + 000010
1100,0001 0000,0010
0xc1 0x02
0xc102
2001:0468:1800:c102::/64
If there is a 2nd point2point link between the same pair of routers,
we could reverse the order of the coreIDs, or use additional bit
patterns in the 2 bits before CoreID2.
Point-to-Point Router Links (Alternate Strategies)
Alternate strategy for numbering p2p links that is less wasteful,
but much more difficult to visually decode:
Internal Subnets
Internal subnets are assigned using the 8 bit SubnetID field. So each
GigaPoP core router can have roughly 256 subnets on attached interfaces
(excluding router-to-router point-to-point subnets). The SubnetID of 0
is reserved for the loopback interfaces.
Loopback Addresses
Loopback addresses are subnetID of 0x00 and InterfaceID of ::0001
Routing Design
In the initial phase of the deployment, the RIPng protocol has been
configured for interior routing. RIPng is a v6-only IGP based on RIP
version 2. It's a simple protocol and it's easy to understand and
debug. Furthermore, since it carries v6 routes only, it has no
interaction with the production IPv4 network. Hence it was deemed a
safe deployment choice. RIPng has the traditional drawbacks of a
distance-vector routing protocol, but in such a small network of routers,
it will hardly be a problem. In the future, OSPF v3 and Integrated
IS-IS protocols will be considered for interior routing. OSPF v3, like
RIPng is for IPv6 only, and runs in a "ships in the night" mode with
respect to OSPF v2 (MAGPI's current v4 IGP). IS-IS is the choice of most
network engineers wishing to do integrated v4/v6 routing.
DNS Configuration
The MAGPI DNS servers support AAAA and IPv6 (long) PTR records. There
are plans to upgrade to BIND9 in the near future and move the
nameservers to IPv6 capable subnets, which will allow us to support
DNS queries over IPv6 transport.
Traffic Accounting and Statistics
[coming soon]
Campus Allocation Ideas
[deferred to a future project]
Future Work
References