How to setup secure access to your local LLM UI
So like many I’ve been playing with local LLMs, here is how I’ve setup secure access to my local instance, using tailscale and my golang light proxy
What you need
- A machine with a suitable GPU (in my case a desktop with a rtx 4090 but could also be an apple silicon mac, an Nvidia DGX Spark, etc)
- Docker (Docker desktop or something compatible)
- Tailscale
- Fortio proxy
That’s it!
Optionally install ollama directly (I did) if you prefer local ollama vs running inside Open Web UI’s image.
Enabling https/TLS certs for Tailscale
In the admin DNS page of the admin console login.tailscale.com/admin/dns, under HTTPS Certificates, select Enable HTTPS (and read the certificate transparency disclaimer).
If you haven’t done so already pick a “fun name” for your ts.net while you’re at it.
See more details or updates in Tailscale’s doc if needed.
Starting the LLM Web UI
From getting started docs:
docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main
This gives you access on http://localhost:3000 but the next step will give https from anywhere on your tailscale network
Starting the https proxy
If you have go
installed you can go install fortio.org/proxy@latest
otherwise get the binary for your host
at github.com/fortio/proxy/releases
Note: I use a binary directly on the host as accessing the tailscale daemon from inside docker isn’t easy, or rather I didn’t know how to map the unix domain socket it expects on linux to the host windows pipe for instance, if you know how, please
let me know, I’d rather use docker run -d for the proxy too (the image is fortio/proxy:latest
if you get it working)
proxy -tailscale -default-route localhost:3000 -timeout 0
Setting timeout to 0 allows slow download of new models (infinite max duration, use -timeout 20m
for 20 minutes for instance)
(ps: on mac/linux you can run it through systemd, or plain nohup proxy &
to keep it running; on windows start /b
etc)
That’s it
Enjoy https://yourmachine.yourtsnet.ts.net/ access!
Comments are welcome: