Many people say 2.4GHz is dead and whilst I view it as ‘best effort’ it isn’t going away and needs to be supported so I want it to be the best it can be. I use a four channel plan on our campus and my colleagues were recently informed this should never be done which inspired this post to explain the decision. This is quite lengthy but hopefully understandable. All comments and criticism welcome.

Continue reading →

I’m in the process of carrying out wifi improvement works on the campus, so I thought I’d write some pieces about our wifi equipment lifecycle and the processes I’m following.

Initial Survey
Pretty much everywhere has some wifi coverage. I’m therefore starting with an initial survey to find out what we currently have and inform what we need to change. Even if I know the coverage is terrible and I need to just start again, this survey is necessary to provide before and after documentation. It also shows how RF propagates through the building, which can help identify troublesome areas of attenuation that don’t show up on the plan. I try to identify where the available data points are at this stage, though it can be hard to be thorough about this.

Design
The design stage is predictive, though it’s sense checked using the initial survey information.

Deploy
Once there’s a new design, this gets deployed – this can be where some compromises occur as it turns out that ceiling cavity contains asbestos, or one of these data cables is damaged. Deployment is sometimes going to be done by me, sometimes by a contractor.

Validate
A repeat of the initial survey – validate the coverage is as expected, that APs are using sensible channels and we don’t have a Co-Channel Interference disaster, that 2.4GHz radios are on a sensible power setting or disabled as is appropriate. At this point there’s a loop back to design in the flowchart. If things are not looking as expected it’s back to the drawing board to find a remedy, then deploy physical or config changes and survey the area again. Recently I’ve carried out validation surveys in areas where I know an install compromise was made because the building structure wasn’t quite as expected. It was strangely satisfying to discover the original design was sound and the new AP position wasn’t working, so I had to come up with a different option.

Monitor
This is an ongoing process, but it’s worth confirming the throughput for group of APs is in the expected range (hasn’t dropped horribly for a building) and retransmissions are at acceptable levels. We use Aruba Airwave which makes it easy to group APs into an area for graphing of basic metrics. When wifi is improved in a building we expect to see an increase in client count and throughput but that isn’t always the case. Improving coverage in a lab means tablets can be used for note taking; that might be one or two more devices, not something that’s going to impact the graph.

I also talk to the users. Let them know there’s been a change, if they don’t already know. Ask for feedback and encourage them to tell you if there’s an device they need to use that doesn’t work so well in some parts of the building. Hopefully the design and validation accounts for this, but…..  I’ll more to day about that in another post.

Just over a week ago I completed my journey towards becoming a Certified Wireless Network Administrator… well, as much as anyone completes any journey in IT, there’s always more to learn.

When I started working in my current job and it became clear nobody understood WiFi, or knew how to solve the problems we were having, I quickly found the CWNP website and the numerous references to CWNA. A year later I was on a CWNA course with Peter Mackenzie. Probably the best aspect of the course was not how much I learned but how much it cemented what I’d already picked up through reading and lab work.

Like most certifications getting a good score in the CWNA exam requires you to learn and remember stuff that in reality you’d look up. In the end I found the exam straight forward. I felt I knew pretty much all the answers (actually it turns out I didn’t know at least three of them), but this is not because it’s an easy exam. I’ve spent a lot of time reading the study guides, making sure I knew what was expected… and then, because I dithered so much between taking the course and sitting the exam CWNA-106 was retired…. so I started again with a new study guide. I was pleased with the end result, a score of 95%. Worth all that work.

I’d like to pursue more certifications, otherwise I’ll need to find something else to do with my evenings. Because I work almost exclusively with Aruba equipment at the moment, it makes sense to look at the ACMA and ACMP certs. I’m also going to go further with the CWNP certs, probably looking at the Analysis Professional next.

The fact CWNP is vendor neutral is particularly appealing. Whilst it’s great to have a certification in Cisco or Aruba wlan equipment (the only two I’d consider worthwhile), these can focus on platform specific commands, licensing requirements and model lineup. Whilst this is useful information it doesn’t teach you about WiFi. I much prefer the approach of understanding how a technology works, something I can apply to any platform, rather than being restricted to knowing how to do things with a particular vendor’s equipment.

Fundamentally, whilst all WiFi network vendors have their pros and cons, their specific technology solutions to problems, RF is RF and if you understand how 802.11 it helps enormously to make sense of why a vendor does something a certain way.

In my experience of the networking world people seem to fall into one of two categories: those that are all about the policy and those that are all about the user.

Actually it’s really IT folk in general and I’m very much in the latter camp. I have on IT worked in schools, non-profits and now at a university and something I’ve found almost all users have in common is they’re not interested in IT policy.

They have a job to do or something they want to achieve. If you start talking about IT policy, as a general rule, the user will see this as getting in their way.

In theory it shouldn’t be this way. Our carefully crafted ITIL compliant framework is all about ensuring users have what they need and a service catalog that’s makes it all clear.

The trouble is often what they want isn’t in the service catalog… so what then? Sure we can spend months developing a solution this fits and rolling it into the catalog, but by then the user has become deeply frustrated, rolled their own solution, or given up completely.

This comes up in our office from time to time. An excellent colleague I respect greatly will often voice something along the lines of “why can’t we just say that’s not supported?”.

Of course in some environments you can do just that. Tell the annoying user they can’t connect their stupid device to the network because it’s too crappy. Sometimes we do. There are some devices that we can’t support on our enterprise Wi-Fi network: Sonos, Philips Hue, chromecast, to name just a few and there are technical barriers to this.

However for the most part we don’t get to choose the device users buy, or control how it interacts with our network. More importantly what is our job? In the university environment we have to maintain a reliable, high performance network that supports the research and education aims of the institution. That means meeting the requirements of the academic departments, finding away to support what they need to do within the technical limitations….. and even working around those where we need to.

When dealing with people who are highly intelligent and resourceful if you don’t provide a way for them to do what they feel is necessary, they’ll find their own way. In doing so they might break things for other people and create a support nightmare further down the road.

I have spent hours messing about with crappy domestic printers bought on the high street on a credit card because I simply hadn’t dealt with issues around printing adequately.

I’m fortunate to work as part of an IT department that has an excellent reputation within our institution. It wasn’t always the case and not very long ago the standard answer to many requests was “no, we can’t/won’t do that”. It has taken years to unpick some of the systems built by departments when central IT couldn’t or wouldn’t help them and they had to find their own way.

I’m not going to suggest the user is always right because we know that’s nonsense. What I will say is put the user first. Your IT policy or administrative convenience is not more important than what the user needs to do.

Let me bring this back to Wi-Fi with an example. Our Aruba system uses centralised forwarding, where all client traffic is tunneled to controllers in the data centre. Some departments have a need for an SSUD that bridges to a local vlan. Right now we can’t support that, because we have to switch on control plane security – a major change. So we could say “no, can’t help” or find a way to do something with an additional controller or standalone AP, and that’s what I did.

You can bet that otherwise we’d see additional rogue APs popping up with departmental SSIDs that would screw with the RF space and be unsupportable. I’d much rather bend, or bin, the policy and keep better control of the network.

I live in a small house by UK standards, most Americans have garden sheds larger, yet getting Wi-Fi to work has always been a challenge.

Domestic Wi-Fi can be difficult, partly because all your neighbours have it too. Chances are in many homes you can pick up dozens of 2.4GHz Wi-Fi networks, which means finding a free channel is pretty much impossible. It’s all best efforts.

It isn’t much better on the 5GHz band. Quite a few ISPs configure their supplied routers 802.11ac routers to use 80MHz channels. They don’t use DFS channels either. Because quite a few of the cheaper 802.11ac routers out there only support the first four channels of 5GHz, you’re out of luck when it comes to finding a free channel.

I have a different problem. My tiny house is a terrace, built in the 1930s. It may be small, and built cheap (a single window spans two rooms upstairs at the front) but boy is it solid. Exterior walls have a cavity with brick on the outside and concrete block inside. The few occasions I’ve had to drill a hole, it’s been very hard work. The non-structural interior partition wall between the two front bedrooms is concrete block, even though it’s just supported by wooden joists.

My ISP supplied router was hopeless. Only really worked in one room of the house. I replaced this with a Draytek unit. As an aside, I really like Draytek routers for domestic or other soho uses. They tend to feel a bit dated now, but my experience is they’re rock solid and work reliably.

However Wi-Fi was still rubbish. In the end I’ve used a Ubiquiti AC-LR access point. Coverage is far superior from the same location (reaches the all important seat in the bathroom), it’s faster and it supports DFS channels so I can keep away from the neighbours.

But, despite this I was having trouble with my desktop PC. This is in the room next to the AP. Draw a straight line between them and it’s less than five metres, yet signal strength is poor and the connection unreliable.

I spent a while messing about with the Wi-Fi interface in the desktop before attention turned to that wall. That single block thickness wall. That very solid partition between two rooms.

Rudimentary measurements show 18dB drop at 2.4GHz and 25dB at 5GHz. This is why I can’t get a decent signal even though I’m not very far away.

The plus side of this is I don’t have a big problem with neighbouring Wi-Fi networks filling my channel space. I appear to live in a tiny faraday cage that’s particularly unfriendly towards 5GHz signals.

The answer will be to relocate the AP to somewhere with less of that special 1930s lead brick between it and my desktop.

Today Nigel Bowden (definitely worth a follow on twitter) drew attention to Ofcom’s March decision to open up more spectrum for indoor wifi use.

UNII-3 contains channels 149-161 which so far have been available for use in the UK for outdoor links, required a license (albeit a very cheap one) and Ofcom allowed TX power of up to 4w. This was particularly useful for linking remote buildings, and I know a few people who make use of this band because there’s no risk of building Wi-Fi interfering with the outdoor link.

From 17 August 2017 it’s all change and this spectrum is also made available, unlicensed, for indoor Wi-Fi use. The restrictions are much the same as the UNII-1 band, channels 36-48, with power limited to 200mw eirp.

For most Wi-Fi network administrators this is most welcome as it gives us an additional four 20MHz channels to play with. For high density networks, and demand for the high throughput capabilities of channel bonding the more spectrum we have available the better.

There was a great deal of confusion about all this. It should be noted the language is quite specific regarding outdoor use:

“Equipment must not form part of a fixed outdoors installation when operating in 5730 – 5850 MHz”

Immediately it looked to anyone with a licensed wireless bridge using these channels it was game over and they’d have to use….. something else.

However that’s not the case, this is just very badly communicated.

The channels remain available for outdoor use at higher power levels only with a license, otherwise they can be used indoors only as low power levels…. phew.

This does mean you need to make sure any external APs are in a group that respects the regulatory domain… Not all vendors do a very good job of this.

So officially we can start using these channels on our indoor Wi-Fi from the 17th of August. Whether your client devices will actually use these channels is a different matter. Odds are it won’t take Apple very long to push out an update to iOS and Mac OS with the new regulatory domain information. It might be a different matter for the large numbers of Android devices we see on our network that never have any sort of update. Either way, there’s some lab work and testing to do before putting the new channels into production use.

For more information, see the Ofcom website. The pertinent document is Interface Requirements 2030. Here is the decision statement from Ofcom.

A recurring problem I face in dealing with wifi performance issues is not being able to relocate access points. Asbestos is a problem in many buildings and in some cases adding extra data cabling is so challenging it just isn’t worth it. But… that isn’t always the reason for things not being quite right.

When wifi was first installed it was expensive. It was also 2.4GHz only. You can stick a 2.4GHz AP in a corridor or communal area between several rooms and it will probably provide pretty good coverage. You have relatively few devices to connect. You want to cover as much of the building as you can, you have no money… what would you do?

So the AP is in the corridor, which isn’t ideal in most cases (a topic for another post). But that’s not the biggest problem…

Our original Colubris, and later some Cisco APs had an external omni antenna that could be turned by 90 degrees. So the AP could be mounted on a wall next to the data socket or on the ceiling. A few vendors and models of AP later, we became an aruba site deployed the AP65 and then the AP125. These have built in antennas but they can be rotated to be the correct orientation when mounted horizontally or vertically. They may look a bit weird, but the design is really flexible.

The problem with this is it means we mount APs on walls next to the data socket, because that’s just what you do. When it comes time to replace the AP125 with something newer we bought the version with a built in antenna, just like before, only now it’s a down tilt omni antenna that’s designed to be mounted on the ceiling. It doesn’t flip round. Worse still the back of the later Aruba APs is metal. So when that gets mounted on the wall in a corridor it doesn’t provide very good coverage to the rear. It’s also putting far more energy to the floors above and below than is likely to be desirable.

We’re creatures of habit and once we know how something’s done, we’ll tend to keep doing it the same way. As a result even as new buildings go up, the APs with down tilt omni antennas get put on the wall rather than the ceiling.

Does it make all that much difference? Yes… it does.

In one building I worked on APs designed to be mounted horizontally on the ceiling were, of course, on the wall just above the suspended ceiling grid. They were nice and neat up there, nobody could see them. Unfortunately not many people could use them either and complaints were routine.

A before and after site survey showed that simply re-positioning the AP in the manufacturer recommended orientation significantly improved the coverage in nearby rooms.

We’re now stuck with APs mounted against the manufacturer’s recommendations in some of our buildings for good reason. You may find yourself in the same boat, and you have to make the best of it. However it’s always worth checking whether or not what you’re about to do still makes sense, even if it’s just the way you’ve always done it.

A couple of years ago I started work as a network technician at a UK university. It was an interesting time to join the department with quite a few staff changes, more cash being spent on a network that was struggling and increasing centralising of IT from academic departments.

There was a lot to do. As a team we were firefighting problems daily from dozens of wifi complaints to broadcast storms and OSPF flaps that would take out chunks of the network.

Another recent recruit set about tackling the core network reliability issues, upgrading ancient firmware and fixing network design issues. We were able to upgrade congested gigabit OSPF links to 10Gig and over time things settled down nicely, ready for a major network redesign that has subsequently been rolled out.

Wifi was particularly interesting. In many parts of the campus it worked just fine, until it didn’t. Often that was in accommodation on an evening, when demand was high. Students are vocal in their disdain for inadequate wifi provision. It’s clear there was a problem with wifi, it quickly became clear to me that nobody really knew what that problem was.

The Aruba system had been installed over a few years with an incredibly small budget. That’s an important point, because it’s easy to criticise a poor design. In this case the colleagues who installed the system managed to provide coverage of almost all of the campus with a fraction of the necessary funding. For the most part it had worked pretty well.

I’ve used wifi for years, since the first ‘affordable’ 802.11b systems were available in the UK but I’d never taken the opportunity to understand it properly. When we tried to fix wifi problems in one accommodation block by doubling the number of APs installed in the corridors, I saw an opportunity to make things better… because what we did just made it a lot worse.

I spent a lot of time reading, some excellent blogs, some of the wifi standards, the updated manufacturer design briefs and started to build a picture of what it takes to design a campus wifi network.

My excellent employer then sent me and another colleague on a CWNA course.

So now, though everyone on the team gets involved, I’ve become the guy who deals with most of the wifi support tickets. I’ve redesigned and re-installed wifi throughout some of the most troublesome buildings as well as designing and deploying wifi in new showcase buildings.

This blog is about what I’ve learned along the way, documenting what we’ve done and why, explaining the compromises we’ve had to make and how much that really matters… and probably some other stuff too. For those with knowledge and experience designing wifi networks it’s unlikely you’ll learn anything new here. I’m undoubtedly more a student than teacher, so feel free to tell me where I’ve got it wrong.

Get in touch on twitter @wifi_zoo