Do you have an idea of how to configure this ?
Do you have an idea of how to configure this ?
To be more explicit: how to configure synchronet such as the original IP is considered for bans, instead of the local ones ?
proxy to provide the actual remote/client IP address to Synchronet. Kind of like this:
https://wiki.synchro.net/howto:haproxy
Self-hosting is such a journey ! Everytime something is broken, I realize it's my understanding of the whole setup that's broken too and I have to learn something new.
With that solved, if I disable synchronet's HA_PROXY, the ws telnet works great. But if I enable HA_PROXY, term is complaining that ws
did not provide enough information and shut downs the connexion.
11/24 09:16:13 term 0040 multisock read_socket() - No data?
11/24 09:16:13 term 0040 * HAPROXY looking for version - failed [fffc01fffb17fffb1c]
Howdy,\o/
Is your syncrhonet running on Windows by chance?Unfortunately not. It's running on raspberrypi4 raspbian under docker.
I am in the process of replacing traefik with haproxy. I got ssh, webv4 and telnet working so far.
ws is a bit problematic because haproxy (from 2.1 https://www.haproxy.com/fr/blog/haproxy-2-1/ ) Defaulted HTTP Representation to HTX . Which means it feeds lowercased headers, and one needs to override this behavior by providing a dictionnary of
headers not to be lowercased, and this insulting option h1-case-adjust-bogus-server
With that solved, if I disable synchronet's HA_PROXY, the ws telnet works great. But if I enable HA_PROXY, term is complaining that ws did not provide enough information and shut downs the connexion.
11/24 09:16:12 srvc 0037 WS Connecting to localhost:23
11/24 09:16:12 term 0040 Working out client address from HAProxy PROTO
11/24 09:16:13 term 0040 multisock read_socket() - No data?
11/24 09:16:13 term 0040 * HAPROXY looking for version - failed [fffc01fffb17fffb1c]
I dont use websockets, so I dont know the extent to which it can be configured.
So reconfigure the websockets part to connect to haproxy's port, not SBBS directly.
The issue is that websocket is a protocol on top of http, that's why we need a websocket->telnet proxy. The issue if I understand correctly is that this proxy is not forwarding haproxy information to the backend.
Or maybe there is a way to configure haproxy directly to act as a websocket proxy for us ?
I gather that HAProxy has a couple of different modes - one for generic TCP services, using its own protocol, and another as a conventional HTTP reverse proxy. Please correct me if I'm wrong on that.
If it sounds like I'm on the right track, I can make a branch that we can test this in. I'd rather avoid setting up HAProxy here, but I can make some changes and you can try them.
at the moment don't need the `check-proxy v2` option for the magic header x-forwarded-for to happen. Instead it just needs the option `forwarder-for`. As far as I understand too yes, I just discovered this mode two days ago.
I just compiled synchronet locally, but if I run it, it complains about missing configuration files. I should RTFM to figure out how to properly do it.
I guess I will be able to pull changes that you push on a branch.
If you want to check if this is really happening, you can put this file somewhere on your Synchronet webserver as 'headers.xjs', then browse to it and see what it spits out:
One of the rows should show the x-forwarded-for header.
Sorry, I should have asked if you did a development / git-based installation in the first place. Once you have one working, let me know. I'll tinker with this a bit tonight if I'm able, or tomorrow.
So reconfigure the websockets part to connect to haproxy's port, not SBBS directly.
The issue is that websocket is a protocol on top of http, that's why we need a websocket->telnet proxy. The issue if I understand correctly is that this proxy is not forwarding haproxy information to the backend.
Assuming it's acting as a typical HTTP reverse proxy where it interacts with the Synchronet web and websocket services, it should be sending the 'x-forwarded-for' header along to the upstream. If so, we would need to make the websocket server:
1) Capture the value of the x-forwarded-for header
2) Check if the telnet server has the HAPROXY_PROTO flag in sbbs.ini
3) Speak the HAProxy protocol when connecting to the telnet server
Parts 1 & 2 are simple. A glance at HAProxy's docs suggest that 3 is no big deal either, but I may be missing something.
Ok, I will focus on getting a git-based installation ready ASAP and let you know. Thank you very much for offering help on that !
Is HAProxy reverse-proxying websocket connections to Synchronet's websocket service? This needs to be configured in addition to reverse proxying traffic to Synchronet's webserver. (Hint: you probably had to set values for 'wsp' and 'wssp' in the [web] section of 'ctrl/modopts.ini' if you set this up.)
You mentioned something in an earlier message about HAProxy lowercasing the names of HTTP headers. If the client sends 'Some-Header: Some Data', can I expect the upstream to receive 'some-header: Some Data'? This will become important, because it looks like the websocket service currently does case-sensitive matching on header lines.
The first is probably easy to do, if you can configure it via a config file (sorry I dont know).
The second would require development I'm guessing...
The first is probably easy to do, if you can configure it via a config file (sorry I dont know).
Unfortunately, there seems to be lots of options for forwarding websocket to a websocket server:
internet-ws -> haproxy -> server ws
but no option for proxying:
internet-ws -> haproxy -> raw tcp with haproxy_proto preamble
browser -> haproxy (http port) -> websocket -> haproxy (telnet port) -> sbbs (telnet port)
Looks like echicken was working on the last item - I would have thought for the first one, you can change "localhost:23" to "haproxy:23"?
Sorry, I should have asked if you did a development / git-based installation in the first place. Once you have one working, let me know. I'll tinker with this a bit tonight if I'm able, or tomorrow.
For your information, Iÿgot a dev/git-based installation working where
Iÿcan easily divert the haproxy traffic to my dev machine with git and all. I could reproduce the issue, so I think Iÿam ready to try whenever you are too.
If I magically got everything right on the first try, then that should do it - but I'm not optimistic that's going to happen. At least we've got something to start with.
I tested your implementation, it almost worked, I had to add padding so that the address length in bytes occupates two bytes (15th and 16th), instead of one, as indicated by the spec.
Can I paste a git patch in SyncTERM ? Is it proper etiquette ?
When I get this whole setup working, I can suggest a few edits to synchronet's wiki's proxy page to account for all those peculiarities.
Good catch - thank you.You are welcome.
I have no problem with it, especially if it's well-formatted for the medium (80x24).Duely noted.
I just pushed a couple of changes: your patch, and a fix because the websocket service was sending the wrong client address. Please update and let me know how it goes.I just pulled, and I confirm it works great for me. I did not a difference for the address fix, as Iÿuse the same computer for dev and testing. I looks good to me though.
I just pulled, and I confirm it works great for me. I did not a difference for the address fix, as Iÿuse the same computer for dev and testing. I looks good to me though.
What is the process to deliver this fix to the docker image ? And how do
you manage synchronet's configuration files with regard to upgrading synchronet ? The way I did it (modifying the files in ctrl/ and overwriting /web from /backups), I feel like I will loose these modifications next time I pull, but maybe I am wrong ?
I have an unrelated issue, should I open a new "thread" for it ? YMODEM-G upload with the ftelnet ws works for very small files (<144ko) but does not work anymore for bigger files (>174ko).
I don't reproduce the issue if I upload files using syncterm.
1) Currently, the websocket service will terminate if the HAPROXY_PROTO option is set, but the X-Forwarded-For header is absent. It won't connect somebody to your terminal server if it can't send their "real" IP address.
the websocket service terminating if the HAPROXY PROTO option is set and the X-Forwarded-For header is absentreally is the *expected and correct* behavior.
2) A bit of testing before I merge this:
2a) If you remove the HAPROXY_PROTO option in sbbs.ini, then restart, do websocket connections (ftelnet) continue to work?
2b) If you remove the HAPROXY_PROTO option in sbbs.ini, restart, and also remove the HTTP reverse proxy from the mix (remember to adjust/remove wsp and wssp in modopts.ini->[web] also) do websocket connections continue to work?
2c) If you leave HAPROXY_PROTO enabled, but remove the HTTP reverse proxy from the mix, do websocket connections fail?
The receiver MUST be configured to only receive the protocol described in this specification and MUST not try to guess whether the protocol header is present or not. This means that the protocol explicitly prevents port
So I think that
the websocket service terminating if the HAPROXY PROTO option is setreally is the *expected and correct* behavior.
and the X-Forwarded-For header is absent
So, instead of connecting at forum.talbot.audio, I connect at localhost:8880 (overriden port for http). Is this equivalent to (remove the HTTP reverse proxy) ?
Okay, so it sounds like various cases are being handled as expected. I'll merge this in later today.
Thanks for your help with testing all of that - I made these changes without actually running any of the scripts myself.
Is your syncrhonet running on Windows by chance?
Unfortunately not. It's running on raspberrypi4 raspbian under docker.
The daily builds of the bbs-io/synchronet image is from git... anything committed should arrive as a new nightly within 48 hours.Sorry, I should have asked if you did a development / git-based installation >> in the first place. Once you have one working, let me know. I'll tinker with >> this a bit tonight if I'm able, or tomorrow.
Ok, I will focus on getting a git-based installation ready ASAP and let you know. Thank you very much for offering help on that !
So reconfigure the websockets part to connect to haproxy's port,
not SBBS directly.
The issue is that websocket is a protocol on top of http, that's
why we need a websocket->telnet proxy. The issue if I understand
correctly is that this proxy is not forwarding haproxy information
to the backend.
Correct, there are two paths that could address this.
* The websocket connects to haproxy, not SBBS directly.
* The websocket speaks that haproxy protocol when talking with
SBBS directly.
The first is probably easy to do, if you can configure it via a
config file (sorry I dont know).
The second would require development I'm guessing...
You mentioned something in an earlier message about HAProxy
lowercasing the names of HTTP headers. If the client sends 'Some-
Header: Some Data', can I expect the upstream to receive 'some-header:
Some Data'? This will become important, because it looks like the
websocket service currently does case-sensitive matching on header
lines.
Okay, so it sounds like various cases are being handled as
expected. I'll merge this in later today.
Thanks for your help with testing all of that - I made these
changes without actually running any of the scripts myself.
Are you running the bbs-io image, or something you cooked up?Yes, I am running the nightly from docker hub.
Only asking as I was considering changing on how the volumes are done... instead of having each subdirectory under sbbs mounted, simply mounting /sbbs-files and the "init" at startup would symlink from /sbbs (this way /exec in particular is still preserved/original and the rest wouldIt sounds like a good idea to me, even if I don't understand what would be the benefits apart from saving some lines of volumes mount in docker-compose.yml ?
largely work the same.
| Sysop: | Nelgin |
|---|---|
| Location: | Plano, TX |
| Users: | 606 |
| Nodes: | 10 (1 / 9) |
| Uptime: | 59:28:49 |
| Calls: | 9,630 |
| Calls today: | 7 |
| Files: | 16,065 |
| Messages: | 1,063,847 |
| Posted today: | 6 |