synapse-haproxy container in CrashLoopBackOff state

Issue 
 We are seeing 
 [karl1@element ~]$ kubectl get pods -n element-onprem
NAME READY STATUS RESTARTS AGE
server-well-known-8c6bd8447-fts78 1/1 Running 2 39h
app-element-web-c5bd87777-745gh 1/1 Running 2 39h
postgres-0 1/1 Running 2 39h
instance-synapse-haproxy-5b4b55fc9c-jv7pp 0/1 CrashLoopBackOff 40 39h
instance-synapse-main-0 1/1 Running 6 39h
 
 and the synapse-haproxy container never leaves the CrashLoopBackOff state. 
 Environment 
 
 Red Hat Enterprise Linux 8.5 
 Element Enterprise On-Premise PoC Installer 0.6.1 
 
 Resolution 
 Add the following lines to /etc/security/limits.conf: 
 * soft nofile 100000
* hard nofile 100000
 
 and reboot the box. After a reboot, the microk8s environment will come back up and the synapse-haproxy container should run without error. 
 Root Cause 
 Check the logs of synapse-haproxy with this command: 
 kubectl logs -n element-onprem instance-synapse-haproxy-5b4b55fc9c-jv7pp
 
 You will want to replace the instance name with your specific instance. See if you have this message: 
 '[haproxy.main()] Cannot raise FD limit to 80034, limit 65536.'
 
 If so, you have run out of open file descriptors and as such the container cannot start.