Secure Proxmox Install – Sudo, Firewall with IPv6, and more – How to Configure from Start to Finish

On Thursday I released an article detailing how to get Proxmox setup and also how to configure networking with IPv6. However that article got long and I just said I would address the firewall in the future. Well, that’s today because I need to get the configuration stuff written down before I forget. In addition to the firewall there are some other security house keeping items for a new proxmox install, that includes disabling the root account and using sudo and changing the default SSH port. So let’s go.

The base OS under Proxmox is Debian. Debian is great and it is lighter-weight than Ubuntu so I am all for using it.


If you are already somewhat comfortable with Proxmox and Debian configuration and just prefer I get to the point then click here to expand this section of text and skip reading the rest.

However something introduced as a default into Ubuntu was the use of “sudo” to elevate a regular user account to root priviliges as needed. Additionally, this allowed for disabling the built-in root account by default (on Ubuntu systems). If you have a public server and you have SSH running on the default port of 22, check your auth.log sometime… The “root” account gets hammered by brute-force attempts all day long. So disabling it all together is a wonderful idea as is switching the default SSH port to something else to avoid script-kiddies roaming the internet for vulnerable boxes with which to play.

This isn’t just “debian” though and the more I work with Proxmox the more I am learning THIS IS PROXMOX. What do I mean by that? Well, disabling the root account has some implications, namely losing admin access to your Proxmox portal. So, before you disable the root account you need to not only setup a new account with SUDO privileges, but you also need to add that user as an admin in Proxmox. Let’s do all of that now…


Setting up a New User and Giving them Sudo Privileges

I will be setting up and account for fictional Jane Doe, eg. jdoe.

SSH to your Debian Install, then…

Fill in all the prompts, you can just ignore full name and room number, etc… Once the user is created add them to the sudoers group.

Then try to SSH to the server and make sure you can login as jdoe and once logged in make sure you can “sudo su” – i.e. elevate yourself to root. If that all works, continue…


Make the User a Proxmox Admin
In the ProxMox Web GUI.

1. On the left, click DataCenter –> Groups tab –> create –> Name: “Admins” –> create
2. Datacenter –> Permissions Tab –> Add –> Group Permission –> Path: / , Group: Admins, Role: Administrator –> Add
3. Datacenter –> Users Tab –> Add –> User name: jdoe, Realm: Linux PAM, Group: Admins –> Add

Now logout of the ProxMox Web-GUI and then log back-in as jdoe using the password you set in the shell earlier. If that all works and Jdoe has permission to do stuff across your node(s) and datacenter you are good to go.

Back in your SSH session… let’s go ahead and disable the root account on your server… Login to SSH using the jdoe account you setup.

Great, now if you try to login with your root account it just plain won’t work. That goes for the proxmox web console as well.


Change Default SSH Port
Next, lets change the default port of SSH. DISCLAIMER – THIS ASSUMES YOU HAVE NOT ENABLED THE PROXMOX FIREWALL YET!!

I will be using the port 9022 as an example…

SSH into your server, then…

This should open up your SSH configuration file in VIM for editing. Find the line that says “Port 22” and underneath that add another line “Port 9022” then save and close the file. The two lines should look something like this:

After you have saved the file, restart the SSH service on the server.

DO NOT close your SSH session window yet. Rather, open up a new SSH shell on the new port to ensure it works. If it does, great, from the new session window:

This reopens the sshd config file for editing. This time, comment out the line “Port 22”. So the two lines should look like this:

Save and close and then restart the SSH service on the server again.

Leave your current SSH session open and make sure you can still establish a new session on the new port and conversely cannot establish a connection on port 22.

Once that is all said and done and working well you should be good to go.

Configuring the ProxMox Firewall

As stated in a previous article, if you are using ProxMox, it IS your firewall. Don’t mess with UFW/iptables on either your proxmox host server or any of your guest systems. Everything most people need from a firewall can be configured via the GUI.

“Cascading Zones”
ProxMox divides the firewall into three “cascading” zones. The top-level is the data center. The middle tier is each Proxmox node in your cluster, and the final tier are the guest systems themselves. I call them “Cascading” tiers because rules in a higher-level tier (**are supposed to**) “cascade” down to all the tiers below… As long as those “tiers” have the firewall enabled. However this described behavior (in the official proxmox documentation) isn’t inline with my experience… See “quirks” below…

Unique Settings, Options, Etc. Per Zone
Configuration across the three different zones differs a bit so I wanted to talk quickly about what is unique in each “zone”

Datacenter – Define “security groups” (groups of ports/services) that can be used in all rules in all zones, define aliases in this zone, define IPSets in this zone. Define default input and output policies.

Node – Notable for NOT being able to define security groups, aliases, or IPsets.

Guest – Define Aliases and IPsets and default input and output policies. Must enable the firewall on each interface in addition to enabling generally.

Quirks
There are several of what I will term “quirks” regarding the Proxmox firewall that might throw you off if you are used to configuring other enterprise firewalls.

1. If a field in a rule is left blank that is the same as saying “any” or “all” except for the “Macro” field which is special…

2. Proxmox comes with predefined “Macros” – Basically instead of saying you want to ACCEPT ALL IN TCP 443, you say you want to ACCEPT IN and then select HTTPS from the Macro list and don’t fill in anything else. Macros aren’t really a quirk per se’. I just wasn’t sure how they worked or what they did without a bit of digging. Basically they provide a way to quickly apply rules, especially for lesser known service ports, without having to look up what you need.

3. Because leaving a field blank = “any” or “all” you can pretty much just define a direction, an action (ex. “allow”), and a Destination Port (ex. 80) and that makes a rule.

4. You will have all kinds of issues getting rules to work on Guest systems if you don’t enable the firewall on all the interfaces on the guest for which you want rules to apply. This really threw me for a loop because I had the firewall “enabled” on the guest but hadn’t explicitly enabled it on any interfaces which led to odd behavior. This ONLY applies to guest machines and not to nodes.

5. The default Input/Output policy REALLY confused me because I am used to just defining this in the rule chain. Default policies are configured under Guest –> Firewall Tab (up top) –> Options Tab (down below) –> Input Policy, Output Policy. I am understanding these to be a “default drop” or “default accept” unless there is a rule saying otherwise for a specific service.

5. While we are on the topic of the default Input Policy,Output Policy settings, I will also note that the ONLY thing that can supersede these defaults are rules on same Guest chain.

6. Which leads to point 6 – When you think all of the above through, it means cascading doesn’t seem to matter. If the default policy on a host is set to “Accept” then it overrides what I would assume are policies that would say for example “deny 22” but the guests default is to allow and that wins. Hence cascading doesn’t actually exist. Feel free to straighten me out if I am wrong on this.

7. The order of rules in an individual chain DOES MATTER on the guest machines. However for the Datacenter level firewall it is hit or miss. I am used to working with firewalls where a rule that is higher in a chain supersedes all rules below it. So, for example, if you you a policy to “DROP” all traffic to your host, every ACCEPT rule below it is null and void. That isn’t always the case with Proxmox based on my testing. Solution? Treat every chain like the order DOES matter and in doing so hopefully avoid unhappy surprises…

A Basic Secure Configuration
All that to say, at the end of the day we mostly just want a “basic” secure configuration so I will attempt to walk through that setup. Before I start though, a disclaimer (yay…)

I am not responsible for you jacking up access to your proxmox server. If you follow these instructions carefully and in order you should be fine. However I can’t emphasize enough the need for care. If you, for example, turn the firewall on and don’t adjust the default rules you will immediately lose all access to your box. So take care!

First, Configure the DataCenter level… (be sure to do these steps IN ORDER). In the Proxmox Web Console

1. Select Datacenter –> Firewall Tab (up top) –> Security Group Tab (down below) –> Create –> Name: proxmox –> create
2. You should now see a new security group called “proxmox” – click it.
3. Now start adding rules with the Rules button. Each rule consists of the following: Direction: In, Action: “Accept”, Tick enable, Protocol: TCP, and a destination port.
3A. There are a total of SIX TCP rules for this security group. Here is what should go in the destination port box for each: 3128, 22, 111, 85, 5900:5999, 8006
4. Still in the security group called “proxmox” add one more rule consisting of the following: Direction: In, Action: “Accept”, Tick enable, Protocol: UDP, and Dest.Port: 5404:5405.

You should now have a total of 7 rules in the proxmox group.

1. Datacenter –> Firewall Tab (up top) –> Rules Tab (down below) –> “Insert:Security Group” –> select “proxmox” and tick “enable” –> add

2. Still on the Datacenter-Firewall-Rules Tab –> Add (to create a new rule) –> Direction: In, Action: Accept, Tick Enable, Protocol: TCP, Dest.Port: 9022 (or whatever you set your SSH port to earlier) –> Add

3. Still on the Datacenter-Firewall-Rules tab, create a new policy with “add” –> Action: Drop, DO NOT TICK ENABLE –> hit “add” –> drag this rule to the bottom of the chain and do not enable it yet.

4. Still on Datacenter –> Firewall Tab –> Options Tab (down below) –> Set input policy to “Accept” —> Set Enable Firewall to “Yes”

5. Still on Datacenter –> Firewall Tab –> Rules Tab (down below) –> tick the box to enable the Drop policy we put at the bottom of the chain in step 3.

Okay, so your Datacenter firewall is now configured. Move on to your Node.

1. Node –> Firewall Tab (up top) –> Rules tab (down below) –> insert: security group –> choose proxmox and make sure it is enabled –> add in a rule for your SSH port that you set earlier and enable it. –> add in a drop rule but don’t enable it –> drag the drop rule to the bottom of the chain, then enable it. Once you are done your node rule chain should match datacenter rule chain.

2. Still on the Node –> Firewall Tab (up top) –> Options tab (down below) –> Set Enable Firewall to “yes”

Your node is now protected.

Finally your guest firewall(s).

1. Guest (100 is the first default guest) –> Firewall Tab (up top) –> Options Tab (down below) –> Set enable firewall to “yes”, Input Policy: Accept, Output Policy: Accept.

2. Guest –> Firewall Tab (up top) –> Rules Tab (down below) –> Add a default drop policy and enable it. –> Add in any other rules you need (ssh port, port 80, 443, etc.) and enable them. Make sure they are all above the drop policy in the chain.

3. Guest –> Network tab (up top) –> double-click on the network connection you want to protect –> ensure “Firewall” is ticked –> hit ok.

Your guest is now protected.

What is the thinking? By setting the default input policies in the “options” tabs at the Datacenter and Guest tiers to “accept” and then manually adding a “drop all” policy to the bottom of each we pretty much are now managing the firewalls all individually. I went this route because I can see what is going on and Cascading just didn’t seem to actually happen correctly (or, if ever) and seemed best to just avoid it.

The documentation for the Proxmox firewall, imho, really just sucks. I burned a lot of hours and brain cells only to come to the conclusion that Cascading doesn’t really happen. Going this route allows me to feel comfortable because it dependably works. I know the ports I want to be closed are closed and the ones I want open are open.

What about IPv6

If you followed my previous article and setup IPv6 correctly then you should be all set. This was all done on Proxmox 4.1 which has full support for IPv6 and most of the kinks seem to be worked out. There is one caveat if you are having trouble with the firewall and think it is related to IPv6, namely you need to have a loopback address for IPv6 defined on your node. It should be defined by default if you setup IPv6 properly. If you want to quickly check then SSH into your node and:

If you see “inet6 addr: ::1/128” then you have an IPv6 loopback address setup.

The firewall TCP and drop rules all apply to both IPv4 and IPv6, you do not need to setup separate rules to apply to one or the other.