Make a Docker Host Fast and Easy with VMware ESXi and Photon OS 3

January 24, 2020
Vernon Keenan
How To
1 Comment

If you’re a up-and-coming tech startup like Taxnexus, you can’t afford to spend all your money on AWS doing devops.

The next time you get stuck with a $500 AWS surprise because someone was really trying to make things work better, think about building a devops playground on-prem or at a local colocation facility.

Move some of your Docker workload over to a bare-metal setup using VMware ESXi, the oldest free, commercial hypervisor. Just imagine all the cheap cores at your disposal with a new AMD Ryzen-based server! And, by using Photon OS as an ESXi-optimized host OS you get the best performance and super-simple, built-in Docker support.

Let’s get started!

Install VMware ESXi and Photon OS

*Hit your new VMware ESXi host on HTTP to access the management tools*

Set up your server hardware with as many cores, memory and fast storage as you can afford. Check this article for more on free ESXi limitations.
Set up ESXi on the local console.
Install your new server in a private network available to your workstations, and then access the management web page to access the VMware Host Client.
Download the Photon OS 3 ISO from the VMware Github repo. These instructions are for the ISO version only; do not use the OVA version.
Upload your ISO to a folder in your VMware datastore.
Create a new VMware virtual machine from the ISO.
Install Photon OS 3 as your first Docker host. Be sure to name your new server!

Now we get to the tricky stuff that kind of makes Photon a pain because is comes up secure and lacking in network nicetities. I use Photon as a single root user, so that requires some additional setup to have a remote SSH work properly.

Set up static IP
Allow external hosts to ping
Enable remote root login
Start and Enable Docker

Set Up Static IP

Access the virtual console in the VMware Host Client and log into your new VM using the root password specified during setup.

To change the IP address from DHCP to static…

# Edit network config file
vi /etc/systemd/network/99-dhcp-en.network

For a host with IP 10.0.200.20/24, DNS and gateway at 10.0.200.1, and in a “mydomain.local” DNS zone change the file to this:

[Match]
Name=e*

[Network]
Address=10.0.200.20/24
Gateway=10.0.200.1
DNS=10.0.200.1
Domains=mydomain.local
NTP=pool.ntp.org

Make sure you have the security right, restart networking and check if you have the new IP active.

# set up security, restart networking and show interfaces
chmod 644 /etc/systemd/network/99-dhcp-en.network
systemctl restart systemd-networkd
ifconfig

Set Up External Ping

If you’re like me, then you like to know when your servers are up by having them send back a reply to an ICMP Echo request. Here are the steps for that:

# change and save iptables
iptables -A OUTPUT -p icmp -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables-save >/etc/systemd/scripts/ip4save

Enable Remote Root Login

The ssh daemon does not allow for remote root login by default. If you are OK with not creating special system users, then you need to enable root login by changing “PermitRootLogin no” to “PermitRootLogin yes” in the daemon config file.

# edit ssh daemon config
vi /etc/ssh/sshd_config

# search for "PermitRootLogin no"
# located at line 125
# change it to this
PermitRootLogin yes

# restart sshd
systemctl restart sshd

Start and Enable Docker

The real glory of this procedure is that Docker comes pre-installed in Photon OS, so you avoid all that mess.

# update to latest docker version
yum update -y
# start docker for the first time
systemctl start docker
# enable docker to start automatically
systemctl enable docker
# check that it is working
docker info
docker run hello-world