The current state of the world has caused some unique stresses on IT infrastructure. For IT departments servicing internal teams, remote access infrastructure in particular has felt the brunt of the blow. To that end, I spent a couple of weeks testing out enterprise VPN solutions.
The need to be able to quickly spin up and rapidly test a variety of solutions had me looking at Azure for research and development.
We had the Following Criteria
- Reliable
- Fast – Must provide a low latency, high throughput connection to ensure a good end-user experience
- Multi-Factor Enabled – Azure MFA was preferable but any standard MFA solution is acceptable
- Control which end-user devices can connect
- LDAP enabled
- Authentication w/ MFA challenge every time a user connects – NOT an “Always-On” VPN solution
- Must be an actual “VPN” solution (see more below)
- Needs to be able to run in Azure. I.E. software running on an OS in a VM or an Azure “virtual appliance”
We looked at Sonicwall’s SMA 500v Virtual Appliance and OpenVPN Access Server running in an Ubuntu 18.04 VM. Beyond these two solutions, we also tested out Azure VPN w/ Azure VPN Client, Windows Server Routing and Remote Access (running in a Windows Server 2019 VM on Azure), and Windows Server Remote Access Gateway (which is not an apples-to-apples comparison because it isn’t a VPN). We integrated all of these solutions (except Azure VPN which has native tie-in to Azure MFA) with a Windows Server NPS server (running in a VM in Azure), LDAP authentication with a pair of domain controllers, and used the Azure Authentication NPS module to integrate with Azure MFA.
How Things Shook Out
- All solutions proved to be fairly reliable. All solutions supported some form of acceptable MFA. All solutions were LDAP enabled. Sonicwall recently released a Virtual SMA appliance for Azure which is almost identical to their on-premise physical SMA appliances. Hence all solutions also met criteria #8.
- Azure VPN, even when used in combination with Conditional Access, didn’t consistently prompt for credentials and MFA. So it was ruled out based on criteria item #6.
- Remote Desktop Gateway – was really cool and met most of the criteria once properly setup. However it was insufficient because we needed more than just RDP for our end-users. It was ruled out due to criteria item #7, it wasn’t a full VPN solution. Which is ideal imho if you don’t need full VPN.
- Windows Server Routing and Remote Access – was the most painful/complicated to setup and even when integrated with NPS it didn’t satisfy criteria number #4. What we discovered is that most VPN solutions that integrate with RADIUS do not properly pass device/computer properties (like the status of being domain joined) through to the NPS server.
- Sonicwall’s Azure SMA virtual appliance was relatively easy to setup (granted, we have a lot of experience with Sonicwall software and I will be the first to admit their documentation sucks, so this may have just been our own interpretation). It met (almost) every single item on the list and was by far the most “full featured” solution we looked at. Spoiler, it lost in the end, more below…
- OpenVPN Access Server – I really wanted the SMA’s to work out but Access Server is where we landed. It met all criteria with some (acceptable) caveats and was a speed demon.
OpenVPN Access Server Stomps Sonicwall’s SMA 500v
Many people are aware of OpenVPN “community edition” which is a free, open-source VPN solution. “Access Server” is a different beast in the same family from the same organization that makes the community edition. Whereas “community edition” is in my opinion a bit of a pain to get stood-up and requires Linux chops to administer, “Access Server” is an enterprise offering (at a pretty reasonable price point) that still requires a bit of Linux know-how but is very easy to stand-up and simple to administer after the fact. If you have a basic handle on administering Linux via the command shell and the technical background to be doing all of the networking and setup involved in deploying any enterprise VPN solution, you should find OpenVPN Access Server a breeze.
I mentioned above that Sonicwall was by far the most full-featured solution. If you need relatively granular and robust security controls, SMA is a great way to go. I would argue that despite the GUI it is more of a pain to setup and configure than Access Server. However, once setup it is relatively easy to administer once you adapt to an incredibly unintuitive interface (i.e. Sonicwalls are weird).
A side note, Sonicwall launched a virtual appliance for Azure several years ago under the the NSv product name. I looked into this in 2018 and discovered that it was buggy, old, and apparently abandoned. If you are pursuing Sonicwall virtual appliances for VPN in Azure, avoid the NSv and go for the SMA 500v. The SMA 500v is running an almost identical firmware/operating system to the on-premise SMA hardware devices. It is routinely updated as of the date of this article, stable, and easy to manage if you are already familiar with their other SMA lines.
So with all that praise given to Sonicwall, why OpenVPN Access Server? In short… SPEED. I have noted that with Sonicwall SMA hardware appliance, total device aggregated VPN throughput rarely exceeds 40 Mbits. I believed it was our networking configuration but could never crack it and suspected it could just be whatever proprietary VPN software was under the hood. I felt a bit validated after witnessing the test results returned by the Sonicwall SMA Azure appliance as we noted similar behavior. Please feel free to correct me here if you have had a better experience.
Testing Environment
- OpenVPN Access Server VM and Sonicwall SMA 500v Azure Appliance were both set to a Standard_B2s size VM (2 vCores, 4 GB RAM).
- We also had a B2s size windows server in the same VNET as both virtual appliances. All three systems in the same Azure Region, subscription, etc.
- To make sure that B2s size systems would not prove to be a network bottleneck, we ran Speedtest.net, Google Speedtest, Bing Speedtest all Directly from a Browser running on the desktop of the B2s sized Windows server mentioned above. The results varied a bit across the different tests but consistently showed anywhere from 500 Mbit to 800 Mbit upload/download speeds from the Windows Server VM. In the case of Speedtest.net (which shows you which location you are testing to/from) we noted the end location it was testing against from its home in Azure. The B2s system and Azure VNET networking were not deemed to be a bottleneck based on these results..
- Both the SMA Firmware and OpenVPN Access Server were fully updated with latest firmware/software. Ditto for the client software on our endpoints.
- Myself and another Admin ran similar speed tests while connected to both the Sonicwall SMA virtual appliance using Netextender, and the OpenVPN Access Server using the OpenVPN Connect Client version 3.x. As we operate our connections as “fully tunneled” this should mean that all traffic from our endpoint is crossing over the VPN tunnel, through the appliance in Azure, and then out to the testing location selected by the speed test.
- I operate off of a fiber connection, the other admin operates off of DSL. My “wall” WAN connection speed is 1 Gbit UP/DOWN but limited to 250 Mbit to my workstation by a hardware firewall appliance. My other admin has 100 Mbit down, and ~20 Mbit up.
Results
The OpenVPN Access Server speed test won the benchmark… BY A LOT. When using Speedtest.net we could verify that the endpoint we were being connected to was the same endpoint connected to from the Windows Server Speedtest.net test. OpenVPN Access server routinely provided numbers around 90 Mbit down/200 Mbit up. The upload speed was basically what I would expect to be full link speed when one considers all the hops involved between my PC and the test endpoint. Download speeds weren’t nearly as spectacular, granted, but when compared to what I will share from Sonicwall they are wonderful. The other admin, with his DSL connection, routinely pulled around 50 Mbit down, and 20 Mbit up on OpenVPN Access Server, again verifying the same testing location endpoint.
Running the same tests while connected to the Sonicwall SMA 500v in Azure, I was able to achieve around 20 mbit Down/23 mbit up. Remember, this is as apples-to-apples as I can think to make it. I even tried to give the Sonicwall an edge (just in case) by sizing the appliance up to a larger D-series VM SKU in azure. It made no difference. The other admin that was testing with me had results of around “15 – 19 Mbit” up/down.
We did test both solutions several times, on several different days. There was some variance in results but the delta between the Sonicwall and the OpenVPN Access Server was almost always the same. The Access Server just beats the SMA Virtual Appliance handily every time.
But I can’t stop there without mentioning…
Support isn’t great for either company. It’s pretty much in line with all of my other vendor support experience. Sonicwall has perhaps a slight edge in this area but at the end of the day, like all vendors, there is little to no sense of ownership for issues or for the products involved. I brought up the performance issues with them, supplied all requested information, diagnostic dumps, etc. and, unsurprisingly, got nowhere. I am pretty convinced it is an inferior product and short of Sonicwall focusing on improving performance, a support ticket just isn’t going to go anywhere.
With OpenVPN access server, they support using RADIUS but unlike every other product tested, there is no way to adjust RADIUS timeout values. In short, it doesn’t work with Azure MFA and again, support wasn’t helpful beyond confirming there was no way to adjust RADIUS timeout values. If your require Azure MFA for authentication, look elsewhere (unfortunately). If you just need MFA, they have built in OATH token support and it works fine; it just isn’t nearly as seamless for end-users.
Limiting which devices can connect to OpenVPN Access Server
To satisfy criteria #4, OpenVPN Access Server has CLI controlled add-in modules for MAC filtering against client devices. This is painful to administer but highly effective. It is something I wish they would integrate into the GUI but it meets the requirement. Sonicwall gets props in that they generate client tokens and have a very robust and easy to manage system when it comes to using these for controlling which devices can connect to your VPN appliance. Again, I was rooting for Sonicwall from a feature standpoint but the performance was just abysmal compared to Access Server. End-user experience trumps administrator experience.
Conclusion
I had a fun time testing out all of the different solutions and doing side-by-side testing was possible thanks to Azure and how easy it was to spin up all of the different options. Access Server isn’t a perfect product, however if you need the quickest VPN solution available, I can heartily recommend it. I know there are many other VPN options (physical and virtual) out there from the likes of Cisco, Meraki, Fortigate, and more. If you have throughput values and administrative experience with another solution (good or bad) feel free to share!