Commit Graph
265 Commits
Author SHA1 Message Date
Sergei ArkhipovandGitHub 74a81a986a Merge pull request #446 from runixer/fix/grease-cipher-suite
Fix DPI detection: replace GREASE cipher suite in FakeTLS ServerHello
2026-04-07 07:47:21 +02:00
Constantine bec321d190 Fix DPI detection: skip GREASE cipher suite in ClientHello parsing
Instead of echoing the first cipher suite from ClientHello (which is
often a GREASE value like 0x5a5a), iterate the list and pick the first
real cipher suite. This is what real TLS servers do per RFC 8701.

Production data shows two client profiles:
- 87% send GREASE first, then 0x1301 (TLS_AES_128_GCM_SHA256)
- 13% send 0xc02b first (TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256)

The fix correctly selects 0x1301 or 0xc02b respectively, matching
real server behavior. Fallback to 0x1301 if all suites are GREASE.

Add snapshot test with GREASE as first cipher suite.
2026-04-06 23:16:08 +03:00
appolimp 5f81ae3743 Improve TCP keepalive and idle timeout for mobile clients
TCP keepalive was configured (SetKeepAlivePeriod) but never actually
enabled (SO_KEEPALIVE) on accepted client connections. Go 1.26's
SetKeepAlivePeriod only sets TCP_KEEPIDLE — it does not call
setsockopt(SO_KEEPALIVE, 1). Without SO_KEEPALIVE the kernel never
sends probe packets, so dead connections from sleeping mobile clients
linger until the idle timeout fires.

Replace SetKeepAlive + SetKeepAlivePeriod with net.KeepAliveConfig
(available since Go 1.24) for explicit per-socket control:

  Idle:     30s   (time before first probe)
  Interval: 10s   (between probes)
  Count:    3     (failed probes to declare dead)

This detects dead connections in ~60s instead of relying on system
defaults (tcp_keepalive_intvl=75s, probes=9 → up to 11 minutes).

Increase the default idle timeout from 1 minute to 5 minutes.
MTProto clients send ping_delay_disconnect every ~60s, which resets
the idle timer. The previous 1-minute default created a race: if a
ping arrived even 1–2 seconds late the relay was killed. A 5-minute
window also survives typical mobile sleep periods (phone idle 2–5 min)
where the NAT mapping is still alive and the connection can resume
without reconnection.

Ref: #132
2026-04-04 12:01:33 +03:00
9seconds f4f969e702 Refactor TLS fragmenting 2026-04-01 14:01:24 +02:00
appolimp 38abee7d7f Support fragmented TLS handshake records
DPI bypass tools like ByeDPI fragment a single TLS record into multiple
records to evade censorship. This broke ReadClientHello because it
assumed the entire ClientHello arrives in one TLS record.

Add reassembleTLSHandshake that reads continuation records and
reconstructs a single TLS record before parsing and HMAC verification.
Per RFC 5246 Section 6.2.1, handshake messages may be fragmented
across multiple records — this is valid TLS behavior.
2026-04-01 09:05:24 +03:00
9seconds b6427ee321 More idiomatic Golang 2026-03-31 15:07:01 +02:00
9seconds 1fcec38aea Change IP address set priority
For a couple of releases we use collected IPs as a prioritized source
for connecting to Telegram. But apparently, they work way worse than it
should, and having connectivity to core ip ALWAYS gives better results.
Thus, this PR flips priorities, so users could have auto-update enabled
as a source of secondary addresses, not primary ones
2026-03-31 11:05:49 +02:00
dolonet eedee63143 Address review: use slices.Clone, simplify concurrent test
- Replace manual make+copy with slices.Clone in Snapshot()
- Remove redundant _ = len(data); Snapshot() call alone is
  sufficient to exercise the lock under -race
2026-03-30 16:17:51 +00:00
Alexey Dolotov 73c6a3aa37 fix: tighten ScoutConnCollected encapsulation and add concurrency test
- Move error check before Snapshot() to avoid unnecessary allocation
- Update existing tests to use Snapshot() instead of direct field access
- Add TestConcurrentAddSnapshot to explicitly exercise the mutex
2026-03-30 15:05:50 +03:00
Alexey Dolotov e54d9d60d3 fix: stabilize flaky CI tests
1. Add sync.Mutex to ScoutConnCollected to eliminate data race between
   Add()/MarkWrite() in readLoop and learn() iterating results.
   Introduce Snapshot() for safe read access.

2. Increase bloom filter test size from 500 to 100000 to prevent
   false negatives from random eviction in the stable bloom filter.

3. Use Require().NoError() in TestHTTPSRequest to prevent nil-pointer
   panic on resp.Body.Close() when the request fails.

Fixes #425
2026-03-30 14:50:32 +03:00
Alexey Dolotov 4642546b35 test: add idleTracker and connIdleTimeout tests
Cover shared idle tracker behavior:
- tracker lifecycle (new, idle after timeout, touch resets)
- read/write with data touches tracker
- read retries on timeout when tracker is not idle
- read closes on timeout when tracker is idle
- shared tracker prevents false timeout across directions
2026-03-30 13:21:51 +03:00
Alexey Dolotov 4627910238 fix: use shared idle tracker for relay connections
connIdleTimeout previously set per-direction deadlines independently.
During media downloads the client→telegram direction can be idle at the
application level while telegram→client is actively streaming data.
After IdleTimeout (default 1 min) the idle direction's ReadDeadline
fires, tearing down the entire relay and breaking media transfers.

Replace the per-direction timeout with a shared atomic timestamp that
both pump goroutines update on any successful Read or Write. When a
ReadDeadline fires on the idle direction, we check the shared tracker:
if the other direction was recently active, we retry instead of closing.
The connection is only torn down when both directions are idle for the
full timeout period.

This matches the documented IdleTimeout contract: "if we have any
message which will pass to either direction, a timer is reset."

Overhead: one atomic.Int64 (8 bytes) per connection pair, one
atomic.Store (~1 ns) per Read/Write with data, zero extra goroutines.

Fixes #423
2026-03-30 10:34:16 +03:00
9seconds 018bd2fdc1 Run release build 2026-03-29 22:54:08 +02:00
9seconds d3a090d6b4 Decrease a relay buffer size for MIPS devices 2026-03-29 21:42:26 +02:00
Alexey Dolotov ec271baab0 fix: apply idle timeout to Telegram relay
Wrap both sides of the Telegram relay in connIdleTimeout,
same as already done for domain fronting in #416.

Without this, if a client disappears (network drop, battery dies),
the TCP connection stays formally alive and the goroutine in the
worker pool blocks on io.CopyBuffer indefinitely. Under mass client
disconnects this accumulates zombie goroutines.

Fixes #417
2026-03-29 16:59:30 +03:00
9seconds 0c030646f9 Remove clock goroutine
This is a followup for https://github.com/9seconds/mtg/issues/412 it
makes sense to manage timers inplace instead of creating for new
goroutines: saves memory
2026-03-29 15:39:33 +02:00
Sergei ArkhipovandGitHub 735466b90d Merge pull request #416 from dolonet/fix/domain-fronting-idle-timeout
fix: apply idle timeout to domain fronting relay
2026-03-29 13:50:45 +02:00
Alexey Dolotov f355512aa6 fix: address staticcheck lint issues
- avoid deprecated DefaultIdleTimeout, use time.Minute directly
- simplify embedded field selectors (QF1008)
2026-03-28 22:59:32 +03:00
Alexey Dolotov 289bb283b1 fix: close connection on worker pool overflow
When the worker pool rejected a connection (ErrPoolOverload), the
accepted net.Conn was never closed — leaking a file descriptor and
TCP socket per rejected connection. Under sustained traffic spikes this
compounds the problem: leaked descriptors reduce the capacity for new
dials (including to the fronting domain), accelerating the failure
cascade described in #378.
2026-03-28 22:52:39 +03:00
Alexey Dolotov 836090ebdf fix: apply idle timeout to domain fronting relay connections
Domain fronting relay (for non-Telegram traffic) had no idle timeout,
causing worker pool exhaustion under traffic spikes.

The ProxyOpts.IdleTimeout field existed but was never wired into the
proxy. Now domain fronting connections are wrapped with per-read/write
deadlines reset to the configured idle timeout (default 1m), so stale
or slowloris-style connections are reaped promptly.

Fixes #378
2026-03-28 22:47:39 +03:00
Alexey Dolotov 026ec74dfd Reduce per-connection memory overhead
- Use sync.Pool for relay buffers instead of stack-allocated arrays.
  A [16379]byte on the goroutine stack forces Go to grow it to 32KB
  (next power of two). Pooled buffers keep goroutine stacks small.

- Same fix for doppelganger write buffer ([16384]byte in conn.start).

- Replace idle goroutines with context.AfterFunc in proxy.ServeConn
  and relay.Relay. These goroutines existed only to wait on ctx.Done()
  and close connections. AfterFunc achieves the same without allocating
  a goroutine until the context is actually cancelled.

Net effect: at 3000 concurrent connections on a 1-vCPU/961MB VPS,
the unmodified binary drops 246 connections and falls to 10 MB/s.
With these changes: zero failures, 63 MB/s, 31% lower RSS.

Closes #412
2026-03-28 13:24:39 +03:00
Alexey Dolotov 9dfd992c1d Move cert noise calibration into doppelganger scout
Instead of a separate cert_probe.go that duplicates the scout's TLS
connection logic, measure the cert chain size directly from the same
HTTPS connections the scout already makes.

Changes:
- Extend ScoutConnResult with payloadLen field
- Add Write interception to ScoutConn for handshake boundary detection
- Scout.learn() now computes cert size (sum of ApplicationData between
  CCS and first client Write) alongside inter-record durations
- Ganger aggregates cert sizes across raids and exposes NoiseParams()
  via atomic pointer for lock-free reads from proxy goroutines
- Proxy reads NoiseParams from Ganger on each handshake instead of
  probing at startup
- Remove cert_probe.go, disk cache, and related config options
  (noise-cache-path, noise-cache-ttl, noise-probe-count)

Falls back to legacy 2500-4700 range until the first scout raid
completes (typically within 1-2 seconds of startup).
2026-03-27 16:34:42 +03:00
Alexey Dolotov 80213ad35d Add dynamic cert noise calibration for FakeTLS handshake
The hardcoded noise range (2500-4700 bytes) in the FakeTLS ServerHello
does not match the real certificate chain sizes of many popular fronting
domains (e.g., dl.google.com ≈ 6480 bytes, microsoft.com ≈ 13004 bytes).
This makes the proxy detectable by DPI systems that compare the
ApplicationData size with the real cert chain size for the SNI domain.

On startup, probe the fronting domain's actual TLS handshake size and
use the measured value ± jitter instead of the static range. Falls back
to the legacy 2500-4700 range if the probe fails.

Also adds optional caching of probe results between restarts
(noise-cache-path, noise-cache-ttl) and a configurable probe count
(noise-probe-count) under [defense.doppelganger].

Closes #408
2026-03-26 23:38:58 +03:00
9seconds 8154f65e0e Add validation of telegram connectivity 2026-03-23 18:34:38 +01:00
9seconds 4a8d099aca Remove unused buffer 2026-03-19 17:39:57 +01:00
9seconds feb57004e1 Fix reslicing 2026-03-19 17:39:48 +01:00
9seconds cb436efd87 Avoid double buffering in TLS hot path 2026-03-19 17:37:51 +01:00
9seconds 724904f50d Wait in doppel.Conn if there is anything to write 2026-03-19 15:42:00 +01:00
9seconds a23ae05f3b Remove SyncWrite 2026-03-19 13:47:08 +01:00
9seconds 913a38d13a Show real IP of the telegram endpoint in event stream 2026-03-18 22:05:34 +01:00
9seconds eb7720b11e Fix flaky test 2026-03-16 23:44:06 +01:00
Maks-2012andGitHub 079252d810 Fix preferIPOnlyIPv6 2026-03-16 16:10:38 +03:00
9seconds bb90bcc127 Decrease relay buffer size
Even if it makes sense to have a huge buffers, we do artificial delays
now. In that case we could achieve the same results with a lower buffer.
If not, then we won't send a packet bigger that this value
2026-03-14 14:06:48 +01:00
Sergei ArkhipovandGitHub 58e6dfd603 Merge pull request #361 from 9seconds/docs
Add docs about doppelganger
2026-03-14 09:43:19 +01:00
9seconds e675baa860 Add docs about doppelganger 2026-03-14 09:42:28 +01:00
9seconds 7adfc0352e Have a noise of variable size 2026-03-13 17:08:44 +01:00
9seconds b35c212d65 Have a noise of variable size 2026-03-13 16:39:16 +01:00
9seconds 45b0964afd Fix android ping 2026-03-13 16:20:18 +01:00
9seconds 4138cc6494 Use SyncWrite call to send a packet to proxy 2026-03-13 14:44:56 +01:00
9seconds 33c0fa9bf7 Add SyncWrite method to doppel.Conn 2026-03-13 11:30:52 +01:00
9seconds 21d7522356 Make DRS optional 2026-03-13 11:04:01 +01:00
9seconds ea71fe81b2 Update stats defaults 2026-03-13 08:59:59 +01:00
9seconds cc63e762e3 Use native dialer for scout 2026-03-12 21:47:03 +01:00
9seconds 62cba24071 Use native dialer to communicate with fronting domain 2026-03-12 20:51:02 +01:00
9seconds 94e4179fb7 Add NativeDialer method to mtglib.Network 2026-03-12 19:25:53 +01:00
9seconds ffd3ab03cc Formatting with gofumpt 2026-03-12 19:13:33 +01:00
9seconds 00403e3a94 Remove old faketls package 2026-03-12 19:11:39 +01:00
9seconds 0bfc1ef2d4 Fix double TLS wrapping for noise 2026-03-12 19:07:11 +01:00
9seconds 7aa01dcebe Limit min number of collected stats 2026-03-12 19:07:11 +01:00
9seconds 1c0e847247 Set statistics measured from real life 2026-03-12 19:07:11 +01:00