Anna asked me to post a message describing the approach we have taken to
control bandwidth on the University of Texas at Austin residential
network. I am not advocating this approach for others, simply explaining
what we are doing as requested (http://resnet.utexas.edu/):
Background:
UT Austin is a commuter campus with ~50K students. About 7K live in our on
campus dormitories. In 1996, we installed a residential network, ResNet,
which now has ~7K ports (port per pillow) and charges a monthly fee for
access. We recognized the problems of the traditional academic Internet
model when we built our ResNet (infinite bandwidth at zero cost), but
figured we'd be able to get away with it by addressing the problem users as
they cropped up in hopes that pre-packaged affordable (free) solutions
would present themselves. They didn't. Thanksgiving of 1999, someone
indexed all the open netbios ports sharing out all the mp3s the tikes had
been storing up. There were to many open hosts to address individually, so
we applied a filter, and teased ourselves that the end was near. That
December, and following January, Napster hit our campus, and we applied yet
another filter. The filters were applied to protect the rest of campus
from ResNet's insatiable bandwidth demands. We knew the filters would fail
as applications and ports changed, but we had to buy time.
We had studied our ResNet traffic patterns for years, and knew that 2% of
the users were consuming 50% of the bandwidth (0.2% consuming 15%). In
nearly all the cases we painstakingly investigated over the years, we found
the students running servers, and the users of those servers were from
outside the University. So people unassociated with the University were
consuming the majority of the bandwidth. Almost all the content from the
servers belonged to someone else. So we got a lot of not so friendly
lawyers contacting the University, causing us to spend time investigating
for our Dean of Student's Office in addition to shutting ports down. Being
in the ISP business, we were merely concerned with the amount of bandwidth
used. All the revenues generated did not cover the amount the 2%
used. Clearly we needed a limit. If we limited just the ResNet system,
then those who were not consuming more than their fair share they had paid
for would suffer slow access. Nor could the residents afford to pay the
actual costs of the 2% (which was ever increasing). I could yammer on, but
I'm sure you have heard all the arguments. We needed a system that fairly
allocated bandwidth amongst the residents.
Fair Access Policy:
Since 9/2000 each resident is granted a weekly bandwidth allocation-- which
they are able to monitor on our web pages. When they exceed this
allocation, their port is deactivated until the next week. As they
approach the limit, they are e-mailed a warning message. When they reach
the limit, their port is deactivated, and they are e-mailed an
explanation. Residents can contact the help desk via phone, or, from our
web site on a different machine or lab machine, may request a small
bandwidth grant for the remainder of the week. When the second grant is
exceeded, the port is again deactivated, and they are e-mailed an
explanation. No exceptions are made. The system is fully automated.
Much study and thought went into the bandwidth limits we chose
initially. The limits will change over time as usage changes. The initial
grant is 3.5GB, warning at 3GB, with an additional grant of 500MB. This is
oversubscribed, but works at the present. Many things could be offered in
the future, like usage sensitive billing.
Implementation basics (I didn't write it-- just hounded the person who did):
We had an existing set of databases used to manage our ResNet that
contained a multitude of information: resident information (from the big
database in the sky), resident to room/port mappings (user registration via
web page), cable to room mappings, cable to hub/switch port mappings,
ethernet mac address to hub/switch port mappings (hub/switch traps),
hub/switch port to vlan assignments, ip to mac address mappings (router arp
table, and special sniffing software on switch span ports). This system
already knows how to activate and deactivate ports via SNMP calls to the
hub/switches for a registration system which is how residents register
their ports. We wrote this software in C++ running on a PC under FreeBSD
and utilizing mysql as the underlying database-- some perl for the web
front ends.
We hung a system off to the side that does accounting. Netflow statistics
are collected from the router, summarized, and placed in the same mysql
database. If we didn't already have a management system to be used to map
flows back to the residents based on ip address, this information would
have been pretty useless. The bandwidth monitor cycles through that
database on a regular basis to turn ports on and off. The residents are
presented graphs based on the same underlying database on the web page for
their registered port. Works for us. Unfortunately, we wrote it for
ourselves and it isn't especially portable because it relies so much on our
existing management system and our internal policies. Again, all in C++.
We hope to package up the Netflow portion (sans ResNet database) to run
under Linux. So someone could drop it on generic Linux machine, configure
their router to export netflow, and get traffic reports for all the hosts
handled by that router. We already do that for the rest of our campus to
track the other 35K hosts (gives us a top N we go through manually and
investigate). But, only if we get time-- we have very few staff for a
network of our size.
Results:
We have been able to meet our ResNet bandwidth goals. All users share a
fair amount of high quality bandwidth. On a typical week < 1% of the ports
are deactivated. A number of these are repeat deactivations (their ports
are activated every Sunday at 00:01 hours and deactivated Monday evening
after using up their allocation). We received very few complaints-- the
residents seemed to understand the reasoning and thought the allocations
were more than fair. We are out of the content monitoring business. My
ResNet administrator now has a life and has even taken up cooking lesson
(something not possible when up at 02:00 with a sniffer).
-William
|