Virtual Security

So, recently I've been thinking a lot about how you secure a virtual environment. As more organisations look to consolidate their server costs, both in terms of hardware and maintenance, naturally a lot of them are going to move towards virtualisation. The problem with this is that a lot of people don't seem to think of the security implications of running all your servers in the sunlit uplands of hypervisor-land.

Sure, you're going to save a tonne of cash, and you can take advantage of all the whizz-bang features like migration and load sharing, but you still need to be careful about segmentation. Running your corporate zone servers on the same hypervisor as your extranet and DMZ servers without thinking about the virtual network deployment is certain to lead to problems when you get hacked.

A few months ago, I was part of a team advising a start up company who were running all of their servers in a pair of blade chassis. They wanted to be able to load share between the chassis for resilience and to enhance the performance of the solution by using load balancing based on server load. All very cunning, but unfortunately they'd designed the solution with no segmentation at all. This meant that it was a major pain to try and secure their solution by effectively retro-fitting their solution with some fairly ugly hacks.

Now, I'm not saying that you can't virtualise your solutions onto a bunch of hypervisors and take advantage of all the snazzy performance enhancing features that could provide a really cool solution, but for pity's sake, make sure your virtual network design keeps different security zones segmented. For this reason, my approach would be to design the virtual network very much in the same way that you would design a physical network in a traditional data centre.

Here are a few recommendations if you're designing a virtualised environment:

1. Make sure your external servers are not on the same segment as your internal and DMZ servers. Likewise, segment your DMZ and back-end servers. You can use the virtual network layer in your hypervisor to do this.

2. Ensure that traffic between different zones goes through a modern firewall appliance. This can either be a dedicated physical appliance, or a virtual appliance integrated into the hypervisor.

3. Use VLANs to segment traffic, but do not rely on them. Whilst VLANs do a good job of logical segmentation, physically you're on the device, and you should assume that traffic can leak between VLANs.

4. Where available, use the security features of your chosen hypervisor to segment VMs at the host level. For instance, you can use VShield if you're a VMWare shop.

5. Consider deploying host-based IDS/IPS on your VMs.

If anyone has any comments on this topic, or other methods of securing a virtual environment, I'd love to hear from you.

Slainte Mhath,
Neil

PCI QSA-ness

So, I'm on my way back to Glasgow just now, after attending QSA training in Orlando. Although I found the course a little slow, I met a lot of interesting people. Hopefully by this time next week I'll be a QSA.

I felt the test went well, but going through the course really opens your eyes to the limitations of the DSS. The first one, is that the standard is very subjective and is open to interpretation at almost every level. This is not a Good Thing!

Secondly, where the DSS is less subjective, it often falls short of industry standard, or best practice. Interestingly, during the course there was some discussion about DSS 2.0, although nothing was mentioned that gives me a warm fuzzy feeling that either of these points will be addressed.

I can understand that the council is not there to mandate particular products or services, however there's nothing to stop them approving encryption and hashing algorithms, for instance. The very likely scenario of QSAs contradicting each other also disturbs me. Everyone has their own opinion on what's an acceptable level of risk or their own interpretation of a given standard. Let's put it this way; I wouldn't want to be assessing a client whom my company had already performed a gap analysis for. The chance of coming to a different conclusion from the gap analysis QSA would be too great.

Empty Logs in ACS 5.1

Cisco's latest version of ACS, 5.1, is a massive improvement on previous versions and has come from a ground-up rebuild of the entire server.  I've used it in a recent deployment for TACACS+ AAA on network devices, plus 802.1x authentication of WLAN users.  So far I've found it pretty easy to use, but I was puzzled today, when perusing the logs to try and figure out why switches weren't authenticating users via TACACS, to find empty logs!

As it turns out, this was due to some (or all) of the view processes not running.  To get them running again, log into the ACS console via SSH and use the following command:

ACS/admin# acs start ?
  adclient           Start adclient
  database           Start database
  management         Start management
  runtime            Start runtime
  view-alertmanager  Start view-alertmanager
  view-collector     Start view-collector
  view-database      Start view-database
  view-jobmanager    Start view-jobmanager
  view-logprocessor  Start view-logprocessor
                 Carriage return.
 

Lightwieght APs Registering to WLC over WAN

Recently I was onsite with a client cursing the fact that their shiny new lightweight access points (LAPs) refused to talk to their assigned WLAN controller (WLC).  Nothing seemed to work, even manually assigning the WLC address to the LAP via the command line didn't help.


As a result I spent a frustrating few hours trying to troubleshoot the whole thing.  The LAP could ping the controller's admin interface (the AP manager interface does not respond to ping) and at one point the LAP even registered successfully with the controller.  Overnight it disassociated itself and couldn't see the WLC again.  Head scratching time.


Eventually, having traced the problem all the way back to the WLC, I discovered that although the WLC was receiving the LWAPP join requests, and transmitting a response, the firewall in front of the controlller wasn't seeing the traffic.  Puzzling.  Checking the ARP table on the ASA revealed the issue.  Because the AP manager doesn't reliably respond to traffic, other than LWAPP, the ASA had no ARP entry for the WLC.  Adding a static entry resolved the issue.  


Once I'd found the source of the problem, it was relatively easy to set up a way so that a completely un-configured LAP could automatically pick up the controller's address and register via DHCP:


1. Configure your firewall in front of your WLC, make sure the correct ports are allowed (UDP 12222 & 12223) and that NAT etc will not interfere.  Add a static ARP entry for your WLC's AP manager interface.
2. Set up a DCHP pool with the network address of your wireless control subnet
3. Set the default gateway as the router in the path to the WLC
4. Set option 43 as an IP address option, with the address of the AP manager interface on your WLC
5. Add the router's IP address on the wireless control subnet to the excluded DHCP addresses.


After that, you should be able to plug in any un-configured LAP and it will automatically get an IP address, register with the WLC and download the correct image.  As an aside, if you happen to interrupt a LAP while it's re-imaging itself, it can sometimes result in a corrupted image.  In this case, you should be able to find the old image still in the flash and boot from that.  The download should then work.


So there you have it, a (nearly) foolproof way of setting up LAPs.

MPLS tomfoolery

So, today's quest was to get a client's site down in England up and runnning on their new MPLS backbone with resilient VPNs over ADSL and 3G.  A quest not exactly assisted by the complete lack of kit when I arrived.  Eventually the kit was tracked down and a somewhat truculent courier forced to return with it.

I'm not sure what it is, but for some reason telcos never seem to be able to get any sort of connection right first time.  This time there was a routing issue within the MPLS cloud which took me an age to confirm and get the telco to resolve.  Needless to say, by this point the client had decided to abort the switch over, so although the MPLS is working now, the site is still on the old connection .

Oh and a little tip - when you're turning up on site expecting to configure a 3G connection, make sure your client knows that they need a SIM card for the HWIC!  

The 3G resilience config has been quite interesting to develop, and I'll post sometime next week on how to implement it, once I've ironed out any bugs in my config :)

New Beginnings

So, here we are - new addition to the blogosphere which no doubt is severely unwanted and unnecessary.  I suppose I should intorduce myself first: I'm Niall Mac Ghillie Aindreas (Neil Anderson in English ;)) and I work as a security consultant for a major security MSP.  I hold a number of certifications from different vendors, but I'm always trying to learn more about networks and computing in general.

The blog is mostly supposed to be a place for me to put my notes while I learn more about networking and security.  Hopefully some of what I put up here will be of use to others.  You'll find a heavy Cisco and Linux bias in the posts as these are the technologies I'm most familiar with.

So, for my first post, I'm going to talk about the new laptop that's just been dropped off by a very pauchled courier - must have been all the stairs.  I've got a new laptop because my flat got burgled a couple of weeks ago, and by a happy chance, the thieviing scumbags took a load of old laptops away along with some more valuable stuff (my wife's laptop and the xbox - sob!).  This gave me the perfect opportunity to buy myself a new lappy which I'm now writing on - happy days!

I went with a Dell Studio 1557 from the Dell Outlet store.  For those who don't know, the Dell Outlet is a site which allows Dell to dispose of any items that may have been returned by users or which have been damaged in some way.  My model has a couple of very minor scratches and was sold as "Scratch or Dent".  It also has a slightly older graphics card in it compared to the main site version.

Overall, I'm very impressed with it so far.  It came with Win7 Home Premium pre-installed, which is a minor annoyance.  I'd much rather the box had come without an OS so I could have installed Linux or used my own Win7 licenses, but never mind.  Build quality is excellent and I'm looking forward to installing all my usual apps (although Firefox is already on!).  I was kind of hoping that it would come with the backlit keyboard, but no dice sadly.  The keyboard is excellent for typing on, with a really nice action.

My 1557 came with 4Gb of RAM and a Core i7 processor which should be more than enough for most of its day-to-day tasks.  The hard drive was sold as 320Gb, but shows up in Explorer as a 287 Gb drive - not sure where all the extra space has gone (maybe Win7 hides space used for the OS?) but it should be more than enough for my purposes.  I'm looking forward to seeing what this machine can do with OpenSuSE on it!

So that's my first impressions of the 1557.  Today I'm working on some IPv6 stuff at home, which I'll post later and getting ready for a client site deployment later in the week.  No doubt, posts will come from these activities later.

Slainte Mhath,
Niall