Self-hosting Rustdesk Signal and Relay Servers
For many years my go-to remote support tool was TeamViewer. Sometime in early 2023 or late 2022 TeamViewer started to be much more aggressive about their enforcement of fair use. Although I was only using TeamViewer to support a single remote user in this timeframe there was a day where I was struggling with connection issues and exceeded my allotment of connection attempts and was locked out of my account. For about a year I went without a convenient remote support solution, struggling with the (in my opinion) clunky and inconvenient Google Chrome Remote Desktop until finally I had enough and decided it was time to do something else. Enter Rustdesk.
About 6 months ago I decided to deploy Rustdesk. Although you can use Rustdesk without any self-hosted components the creators do encourage users to set up their own Signal and Relay servers. Rustdesk does provide Signal and Relay servers but they do encourage self-hosting. The process for setting up Rustdesk Signal and Relay servers is very straight-forward. I'm partial to vanilla Debian linux so that's what I went with. I used this documentation: https://rustdesk.com/docs/en/self-host/
If you don't plan to purchase a Pro subscription (I didn't), once everything is installed there really isn't a server web interface to explore. It just looks like this, with a couple of install scripts available to download:

If this is what you see then presumably you've followed the steps correctly and everything is good to go. I'll admit I never use these scripts or this web page. Everything I do is in the Rustdesk Client software. I've successfully used it on Windows, Linux Mint, and Android. The important detail with self-hosting is correctly configuring the client with your server information. Once the client is running (I always install rather than one-time run), go to Settings-->Network-->ID/Relay server and plug in your server information. It will look something like this:

So, when I started this adventure about 6 months ago, I got everything up and running and Rustdesk was working for connections inside my network. So, I could connect from my desktop computer on my wired subnet to my laptop on my wireless network. But, external connections were not working. I scoured the logs and could see the connection establishing, but I was unable to access remote devices and was always met with this error message: "Could not open connection to the host, on port 21110: Connect failed." All of my firewall configurations were right, the devices were healthy, the server was healthy. So, after banging my head for a day or two I decided to leave it and move on to something else.
Fast forward to a couple of weeks ago. I really needed this solution to work for remote devices, so I got back to it. A lot of googling later I stumbled upon this Reddit post: https://www.reddit.com/r/rustdesk/comments/1d9td6u/cant_connect_to_selfhosted_server_outside_the/
Eureka! I had configured all of my internal clients to use the IP address of the relay/signal server but external clients naturally had to use a DNS address (or at the very least the external IP address). Even though all of the clients were communicating with the same server, the configurations were different and the connections would fail.
So, lesson learned: set up your internal DNS and external DNS to use the same FQDN and use that in your client configurations. Even though the internal DNS will be pointed at the internal IP address of your server and the external DNS will be pointed at the external (NAT'd) IP address this is the correct configuration to use.
I hope this saves someone else some time.
-Andrew
topics linux, rustdesk