Score:1

Server

Redirect HTTP to HTTPS whilst allowing ACME challenges through the 'Standalone HTTP server' method in pfSense (with ACME and HAProxy)?

Martin Braun

12/28/23, 12:46 AM

I use my pfSense with ACME and HAProxy extensions to manage and auto-renew certificates as well as having a reverse proxy with load balancing capabilities. In my ACME module I define my domains to challenge for like so:

This means once my certificate will be re-newed, a standalone HTTP server will be launched that will listen on port 80. The 'well known acme challenge' files will be reached on such server and my certificate will be validated.

Now I'd like to redirect all HTTP traffic to HTTPS in the reverse proxy (HAProxy frontend) of pfSense. For this, I could setup a new frontend that listens on the WAN address on port 80 in the HAProxy module that will redirect if the path does not start with /.well-known/acme-challenge/ like so:

So if the path starts with /.well-known/acme-challenge/, no redirect will occur, but if not, the HTTP request will respond with a Location header to the HTTPS version of the requested URL. The ACL conditioning works so far, but my problem is that HAProxy will now return "503 Service Unavailable - No server is available to handle this request." when I request anything within /.well-known/acme-challenge/.

This is obvious: HAProxy started to listen on the port 80 of the WAN address, so my Standalone HTTP server gets shadowed by it. HAProxy can only return error 503 as it doesn't know what pfSense itself would do with such request.

What I think I need to do is to setup a backend that I send the request to, if the request's path starts with /.well-known/acme-challenge/, but how do I have to setup such backend in HAProxy, so it works with the Standalone HTTP server of the ACME module?

I've tried to add a backend for 127.0.0.1:80 as well as the IP of my host on port 80. I then added another Action on my frontend in HAProxy with the Condition acl names acme to such backend, but I still get error 503 and this is not due to the fact that my standalone server is not running yet. I get the same error when I try to run the ACME challenge with the Let's Encrypt Staging Environment.

[ The log says: Verifving; Standalone mode server; Pending, The CA is processing your order, please just wait. (1/30); socat989891 E write(6, 0x800add000, 39): Broken pipe
scat!209211 E write(6. 0x800add000. 126): Broken pipe
.com:Verify error:...: Invalid response from http://.../.well-known/acme-challenge/...

How can I make the ACME Standalone HTTP server as well as my HTTP to HTTPS redirection co-exist without any problems?

972

1 + 4

haproxy

pfsense

lets-encrypt

setenforce 1

1/20/24, 10:56 AM

It might be a binding issue, where both your webserver and haproxy are trying to listen on the same ip:port tuple. You should be able to configure your webserver to listen on 127.0.0.1:80 only (or 192.168.0.1:80), and your haproxy on wan_ip:80 only, so the webserver will start.

Martin Braun

1/20/24, 5:53 PM

@setenforce1 It definitely a port conflict, since the webserver that pfSense spawns for the ACME challenge needs to be reachable through the domain that routes to the wan_ip:80 as well. So I tried to create the routing with my backend on "127.0.0.1:80 only (or 192.168.0.1:80)", though it was simply ignored.

Ginnungagap

1/22/24, 8:59 AM

@MartinBraun, is the use of the standalone web server a hard requirement? If not you can use webroot so challenge files get written to a folder and served by the webserver.

Martin Braun

1/23/24, 12:45 PM

@Ginnungagap Not necessarily. It all boils down to the practicality and the required work-load. I will have to setup a lot of services in the near future and it would help to get a reliable solution that doesn't waste my time long term.

Score:0

Server

Martin Braun

1/26/24, 12:02 PM

I don't know what I did wrong previously, but I re-attempted it and managed to get it to work with the same thought processing in mind. I created a backend that will be my local web server:

My http/https offloader (front-end) defines a path rule and redirects to such backend if we have an ACME challenge:

Maybe it was the ordering of the actions, maybe it was the naming. I was pretty confident that I tested my previous setup with 127.0.0.1 as well, but this seems to work and I don't know why it did not work previously.

Now it was very easy to confirm the configuration is right when using postman. Say you have the domain example.org, you should do a GET request to two different URLs to validate their response:

http://example.org/foobar: Should return a Location header with the https version of the URL, so confirming the offloader works
http://example.org/.well-known/acme-challenge/foobar: Should timeout! It must not return an error immediately, or the configuration is wrong. If the configuration is right, it will try to talk to the standalone HTTP server that only runs during the ACME challenge, so it will timeout with 503 Service Unavailable after 60 seconds or so, which means it will succeed if the standalone HTTP server is running.

With this setup the "Standalone HTTP server" method will work.

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Redirect HTTP to HTTPS whilst allowing ACME challenges through the 'Standalone HTTP server' method in pfSense (with ACME and HAProxy)?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.