Without this, mtg and Caddy see HAProxy's container IP for every
connection, which breaks meaningful logging, abuse handling, and any
IP-based blocklist logic. HAProxy sends a PROXY protocol v2 header on
its TCP backends; mtg enables proxy-protocol-listener, and Caddy wraps
:8443 with a proxy_protocol listener before tls.
The :80 path (ACME HTTP-01 passthrough) is unchanged — client IP there
is not useful and HAProxy's http mode already adds X-Forwarded-For if
anyone wants it.
Requested in https://github.com/9seconds/mtg/pull/462 review.
The previous wording ("silently routed to the fronting domain")
is inaccurate. In mtglib/proxy.go the blocklist path calls
conn.Close() immediately with no further handshake or fronting;
domain fronting only happens on FakeTLS failures for non-blocked
IPs. Reword to "TCP connection is closed with no response" so
users searching the docs get the same symptom they actually see.
The default [defense.blocklist] uses firehol_level1.netset, which
includes bogon networks and therefore all RFC1918 ranges. Clients
connecting from a LAN address (e.g. a phone on the home Wi-Fi when
mtg runs at home) are silently rejected with "ip was blacklisted"
and routed to the fronting domain. This is a recurring source of
confusion (see issue #466 for the latest example).
Add a warning next to the urls list in example.config.toml and a
Troubleshooting section in README.md covering the symptom, the
cause, and three resolution paths (disable blocklist, swap for a
narrower list, or use hairpin NAT).
Docs only, no code changes.
Add an ACL that routes /.well-known/acme-challenge/ requests on :80
to Caddy instead of redirecting to HTTPS, so Let's Encrypt certificate
issuance works out of the box.
Also simplify Caddyfile to use Caddy's http_port/https_port directives.
Turnkey deployment: HAProxy on :443 peeks at the TLS SNI and routes
Telegram clients to mtg while forwarding everything else (including DPI
probes) to a real Caddy web server with automatic HTTPS.
This is the setup recommended in BEST_PRACTICES.md, packaged so that
operators can clone and run it with minimal configuration.
Refs: #458
Fixes#457.
OpenBSD has no user-settable per-socket TCP keepalive options:
TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT do not exist on OpenBSD,
keepalive timing is controlled system-wide via the sysctls
net.inet.tcp.keepidle and net.inet.tcp.keepintvl. Go reflects this in
src/net/tcpsockopt_openbsd.go: setKeepAliveIdle / Interval / Count
return ENOPROTOOPT for any non-negative value, and only short-circuit
to nil for negative values that explicitly mean "leave alone".
mtg builds a net.KeepAliveConfig with zero-valued Idle / Interval /
Count whenever the user does not override them in the config (which
is the default and the documented expectation). It then hands that
config to (*TCPConn).SetKeepAliveConfig in two places:
- network/sockopts.go: applied to every connection accepted by
internal/utils.Listener.Accept and to every server-side dial that
goes through the v1 default network.
- network/v2/sockopts.go: applied to every connection produced by
the v2 network's DialContext.
On OpenBSD both calls fail with "set tcp ...: protocol not available".
The user-visible effect is that:
- `mtg doctor` reports the error for every Telegram DC.
- `mtg run` accepts incoming TCP connections at the kernel level but
Listener.Accept then closes each one before the proxy server ever
sees it, so the client appears to hang on a half-open socket and
nothing is logged.
- There is no configuration workaround. Setting [network]
keep-alive.disabled = true only zeroes Enable; Go still calls
setKeepAliveIdle / Interval / Count, which still fail.
This change extracts the keepalive setup behind an applyKeepAlive
helper that has a per-platform implementation, following the same
build-tag pattern already used for sockopts_lowat, sockopts_congestion,
sockopts_reuseaddr and sockopts_usertimeout. On every supported
platform except OpenBSD it still calls SetKeepAliveConfig and the
behaviour is unchanged. On OpenBSD it calls SetKeepAlive(cfg.Enable)
instead, which only flips SO_KEEPALIVE on or off and never touches
the missing per-socket options. OpenBSD users get the system-wide
sysctl-controlled keepalive timing, which is the only thing the
kernel exposes anyway.
Verified by cross-building (`GOOS=openbsd GOARCH=amd64 go build ./...`
and `GOARCH=arm64`) and by running `go test ./network/...` on linux.
- Replace manual make+copy with slices.Clone in Snapshot()
- Remove redundant _ = len(data); Snapshot() call alone is
sufficient to exercise the lock under -race
- Move error check before Snapshot() to avoid unnecessary allocation
- Update existing tests to use Snapshot() instead of direct field access
- Add TestConcurrentAddSnapshot to explicitly exercise the mutex
1. Add sync.Mutex to ScoutConnCollected to eliminate data race between
Add()/MarkWrite() in readLoop and learn() iterating results.
Introduce Snapshot() for safe read access.
2. Increase bloom filter test size from 500 to 100000 to prevent
false negatives from random eviction in the stable bloom filter.
3. Use Require().NoError() in TestHTTPSRequest to prevent nil-pointer
panic on resp.Body.Close() when the request fails.
Fixes#425
Cover shared idle tracker behavior:
- tracker lifecycle (new, idle after timeout, touch resets)
- read/write with data touches tracker
- read retries on timeout when tracker is not idle
- read closes on timeout when tracker is idle
- shared tracker prevents false timeout across directions
connIdleTimeout previously set per-direction deadlines independently.
During media downloads the client→telegram direction can be idle at the
application level while telegram→client is actively streaming data.
After IdleTimeout (default 1 min) the idle direction's ReadDeadline
fires, tearing down the entire relay and breaking media transfers.
Replace the per-direction timeout with a shared atomic timestamp that
both pump goroutines update on any successful Read or Write. When a
ReadDeadline fires on the idle direction, we check the shared tracker:
if the other direction was recently active, we retry instead of closing.
The connection is only torn down when both directions are idle for the
full timeout period.
This matches the documented IdleTimeout contract: "if we have any
message which will pass to either direction, a timer is reset."
Overhead: one atomic.Int64 (8 bytes) per connection pair, one
atomic.Store (~1 ns) per Read/Write with data, zero extra goroutines.
Fixes#423
Wrap both sides of the Telegram relay in connIdleTimeout,
same as already done for domain fronting in #416.
Without this, if a client disappears (network drop, battery dies),
the TCP connection stays formally alive and the goroutine in the
worker pool blocks on io.CopyBuffer indefinitely. Under mass client
disconnects this accumulates zombie goroutines.
Fixes#417
On some servers ifconfig.co is unreachable (e.g. Hetzner, AdGuard DNS
blocklists), causing 'mtg doctor' SNI-DNS check and 'mtg access' link
generation to fail. New config options allow specifying public IPs
manually, with automatic detection as fallback.
Fixes#405
When the worker pool rejected a connection (ErrPoolOverload), the
accepted net.Conn was never closed — leaking a file descriptor and
TCP socket per rejected connection. Under sustained traffic spikes this
compounds the problem: leaked descriptors reduce the capacity for new
dials (including to the fronting domain), accelerating the failure
cascade described in #378.
Domain fronting relay (for non-Telegram traffic) had no idle timeout,
causing worker pool exhaustion under traffic spikes.
The ProxyOpts.IdleTimeout field existed but was never wired into the
proxy. Now domain fronting connections are wrapped with per-read/write
deadlines reset to the configured idle timeout (default 1m), so stale
or slowloris-style connections are reaped promptly.
Fixes#378
On 32-bit architectures (e.g. ARM7), int is 32 bits wide.
Casting a uint32 hash value to int can overflow, producing a
negative number. Go's modulo operator preserves the sign, so
the channel index can become -1, causing a panic.
Perform the modulo in uint32 space before indexing to ensure
the result is always non-negative.
Fixes#413
- Use sync.Pool for relay buffers instead of stack-allocated arrays.
A [16379]byte on the goroutine stack forces Go to grow it to 32KB
(next power of two). Pooled buffers keep goroutine stacks small.
- Same fix for doppelganger write buffer ([16384]byte in conn.start).
- Replace idle goroutines with context.AfterFunc in proxy.ServeConn
and relay.Relay. These goroutines existed only to wait on ctx.Done()
and close connections. AfterFunc achieves the same without allocating
a goroutine until the context is actually cancelled.
Net effect: at 3000 concurrent connections on a 1-vCPU/961MB VPS,
the unmodified binary drops 246 connections and falls to 10 MB/s.
With these changes: zero failures, 63 MB/s, 31% lower RSS.
Closes#412
Instead of a separate cert_probe.go that duplicates the scout's TLS
connection logic, measure the cert chain size directly from the same
HTTPS connections the scout already makes.
Changes:
- Extend ScoutConnResult with payloadLen field
- Add Write interception to ScoutConn for handshake boundary detection
- Scout.learn() now computes cert size (sum of ApplicationData between
CCS and first client Write) alongside inter-record durations
- Ganger aggregates cert sizes across raids and exposes NoiseParams()
via atomic pointer for lock-free reads from proxy goroutines
- Proxy reads NoiseParams from Ganger on each handshake instead of
probing at startup
- Remove cert_probe.go, disk cache, and related config options
(noise-cache-path, noise-cache-ttl, noise-probe-count)
Falls back to legacy 2500-4700 range until the first scout raid
completes (typically within 1-2 seconds of startup).
The hardcoded noise range (2500-4700 bytes) in the FakeTLS ServerHello
does not match the real certificate chain sizes of many popular fronting
domains (e.g., dl.google.com ≈ 6480 bytes, microsoft.com ≈ 13004 bytes).
This makes the proxy detectable by DPI systems that compare the
ApplicationData size with the real cert chain size for the SNI domain.
On startup, probe the fronting domain's actual TLS handshake size and
use the measured value ± jitter instead of the static range. Falls back
to the legacy 2500-4700 range if the probe fails.
Also adds optional caching of probe results between restarts
(noise-cache-path, noise-cache-ttl) and a configurable probe count
(noise-probe-count) under [defense.doppelganger].
Closes#408
dolonet