REPOSITORY / ScuroNeko/mtg

Compare commits

DIFF REPOSITORY

Compare commits

...
51 Commits
Author SHA1 Message Date
ScuroNeko 98aa31b5f5 removed unwanted log write
doctor logging per-ip
2026-04-20 13:45:03 +03:00
ScuroNeko f32ca9f568 gitignore 2026-04-20 13:06:03 +03:00
ScuroNeko c80d3452f0 support for test dc1-3
loading custom dc ips from config
2026-04-20 13:04:13 +03:00
Sergei ArkhipovandGitHub 9bf7222208 Merge pull request #467 from dolonet/docs/blocklist-lan-gotcha
docs: warn about firehol_level1 RFC1918 gotcha in default blocklist
2026-04-14 14:06:34 +02:00
Sergei ArkhipovandGitHub 5953f9320c Merge pull request #462 from dolonet/contrib/docker-sni-router
Add docker-compose example with HAProxy SNI router
2026-04-14 14:06:18 +02:00
dolonet 170346bb74 Pass real client IPs through with PROXY protocol v2
Without this, mtg and Caddy see HAProxy's container IP for every
connection, which breaks meaningful logging, abuse handling, and any
IP-based blocklist logic.  HAProxy sends a PROXY protocol v2 header on
its TCP backends; mtg enables proxy-protocol-listener, and Caddy wraps
:8443 with a proxy_protocol listener before tls.

The :80 path (ACME HTTP-01 passthrough) is unchanged — client IP there
is not useful and HAProxy's http mode already adds X-Forwarded-For if
anyone wants it.

Requested in https://github.com/9seconds/mtg/pull/462 review.
2026-04-13 07:58:16 +00:00
dolonet 68a4685ec6 Fix description of blocklist rejection behavior
The previous wording ("silently routed to the fronting domain")
is inaccurate. In mtglib/proxy.go the blocklist path calls
conn.Close() immediately with no further handshake or fronting;
domain fronting only happens on FakeTLS failures for non-blocked
IPs. Reword to "TCP connection is closed with no response" so
users searching the docs get the same symptom they actually see.
2026-04-13 07:46:52 +00:00
dolonet 602f85d24d Document firehol_level1 RFC1918 gotcha in blocklist defaults
The default [defense.blocklist] uses firehol_level1.netset, which
includes bogon networks and therefore all RFC1918 ranges. Clients
connecting from a LAN address (e.g. a phone on the home Wi-Fi when
mtg runs at home) are silently rejected with "ip was blacklisted"
and routed to the fronting domain. This is a recurring source of
confusion (see issue #466 for the latest example).

Add a warning next to the urls list in example.config.toml and a
Troubleshooting section in README.md covering the symptom, the
cause, and three resolution paths (disable blocklist, swap for a
narrower list, or use hairpin NAT).

Docs only, no code changes.
2026-04-13 07:08:08 +00:00
dolonet d0412b21f6 Fix ACME HTTP-01 passthrough in HAProxy config
Add an ACL that routes /.well-known/acme-challenge/ requests on :80
to Caddy instead of redirecting to HTTPS, so Let's Encrypt certificate
issuance works out of the box.

Also simplify Caddyfile to use Caddy's http_port/https_port directives.
2026-04-10 10:50:26 +00:00
dolonet 0c1d001949 Add docker-compose example with HAProxy SNI router
Turnkey deployment: HAProxy on :443 peeks at the TLS SNI and routes
Telegram clients to mtg while forwarding everything else (including DPI
probes) to a real Caddy web server with automatic HTTPS.

This is the setup recommended in BEST_PRACTICES.md, packaged so that
operators can clone and run it with minimal configuration.

Refs: #458
2026-04-10 10:12:39 +00:00
Sergei ArkhipovandGitHub d7249756e2 Merge pull request #459 from dolonet/fix/openbsd-keepalive
Fix TCP keepalive setup on OpenBSD
2026-04-10 11:47:21 +02:00
dolonet 2fa0e5ed94 Fix TCP keepalive setup on OpenBSD
Fixes #457.

OpenBSD has no user-settable per-socket TCP keepalive options:
TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT do not exist on OpenBSD,
keepalive timing is controlled system-wide via the sysctls
net.inet.tcp.keepidle and net.inet.tcp.keepintvl. Go reflects this in
src/net/tcpsockopt_openbsd.go: setKeepAliveIdle / Interval / Count
return ENOPROTOOPT for any non-negative value, and only short-circuit
to nil for negative values that explicitly mean "leave alone".

mtg builds a net.KeepAliveConfig with zero-valued Idle / Interval /
Count whenever the user does not override them in the config (which
is the default and the documented expectation). It then hands that
config to (*TCPConn).SetKeepAliveConfig in two places:

  - network/sockopts.go: applied to every connection accepted by
    internal/utils.Listener.Accept and to every server-side dial that
    goes through the v1 default network.
  - network/v2/sockopts.go: applied to every connection produced by
    the v2 network's DialContext.

On OpenBSD both calls fail with "set tcp ...: protocol not available".
The user-visible effect is that:

  - `mtg doctor` reports the error for every Telegram DC.
  - `mtg run` accepts incoming TCP connections at the kernel level but
    Listener.Accept then closes each one before the proxy server ever
    sees it, so the client appears to hang on a half-open socket and
    nothing is logged.
  - There is no configuration workaround. Setting [network]
    keep-alive.disabled = true only zeroes Enable; Go still calls
    setKeepAliveIdle / Interval / Count, which still fail.

This change extracts the keepalive setup behind an applyKeepAlive
helper that has a per-platform implementation, following the same
build-tag pattern already used for sockopts_lowat, sockopts_congestion,
sockopts_reuseaddr and sockopts_usertimeout. On every supported
platform except OpenBSD it still calls SetKeepAliveConfig and the
behaviour is unchanged. On OpenBSD it calls SetKeepAlive(cfg.Enable)
instead, which only flips SO_KEEPALIVE on or off and never touches
the missing per-socket options. OpenBSD users get the system-wide
sysctl-controlled keepalive timing, which is the only thing the
kernel exposes anyway.

Verified by cross-building (`GOOS=openbsd GOARCH=amd64 go build ./...`
and `GOARCH=arm64`) and by running `go test ./network/...` on linux.
2026-04-09 16:06:46 +00:00
9seconds fb94d4a78b Update dependencies 2026-04-07 18:09:39 +02:00
Sergei ArkhipovandGitHub b5799500e4 Merge pull request #454 from 9seconds/tcp-notsent-lowat
Add TCP_NOTSENT_LOWAT setting
2026-04-07 15:45:53 +02:00
9seconds 437dacfaab Refactor socksopts per functionality, not per build flag 2026-04-07 15:42:22 +02:00
9seconds 0de8b28de8 Add TCP_NOTSENT_LOWAT setting 2026-04-07 15:39:26 +02:00
Sergei ArkhipovandGitHub 2f62e8055d Merge pull request #453 from 9seconds/tcp-user-timeout
Add TCP_USER_TIMEOUT support
2026-04-07 15:38:11 +02:00
9seconds b58ac669d5 Add TCP_USER_TIMEOUT support 2026-04-07 15:35:47 +02:00
Sergei ArkhipovandGitHub 60ab083def Merge pull request #452 from 9seconds/tcp-bbr
Use TCP BBR in a best-effort mode
2026-04-07 15:33:27 +02:00
9seconds 88f33debab Use TCP BBR in a best-effort mode
This small commits conditionally sets TCP BBR as a preferrable algorithm
for TCP congestion control
2026-04-07 14:54:56 +02:00
Sergei ArkhipovandGitHub ac40c94820 Merge pull request #451 from 9seconds/keepalive-config
Propagate keep alive settings from the config
2026-04-07 13:53:00 +02:00
9seconds 102f8a6cce Propagate keep alive settings from the config 2026-04-07 13:41:44 +02:00
Sergei ArkhipovandGitHub 83b43aecc1 Merge pull request #448 from 9seconds/no-default-tls-cipher
Do not use default TLS cipher
2026-04-07 12:13:21 +02:00
9seconds 5bf218f9ab Do not use default TLS cipher
As per RFC, if TLS server cannot pickup a suitable cipher from a client
list, it has to send handshake_failure alert. For us it means that we
have to route a request to a fronting domain, because we want to have it
exactly like a real webserver does. So, if it misbehaves, so do we.
2026-04-07 12:08:14 +02:00
Sergei ArkhipovandGitHub efc65f317e Merge pull request #447 from 9seconds/handshake-timeout
Add separate handshake timeout
2026-04-07 08:29:23 +02:00
9seconds 39ab5570a8 Small refactoring 2026-04-07 08:29:05 +02:00
9seconds eb564936c7 Add separate handshake timeout
This PR adds a new setting to the config: `network.timeout`. This setting
defines a time period during which all handshake procedures and
ceremonies must be completed. If not - connection is aborted. This
should help in situations when connection is established but client
cannot continue for some reason (for example, RST sent by some middle box).
2026-04-07 08:01:51 +02:00
Sergei ArkhipovandGitHub 74a81a986a Merge pull request #446 from runixer/fix/grease-cipher-suite
Fix DPI detection: replace GREASE cipher suite in FakeTLS ServerHello
2026-04-07 07:47:21 +02:00
Constantine bec321d190 Fix DPI detection: skip GREASE cipher suite in ClientHello parsing
Instead of echoing the first cipher suite from ClientHello (which is
often a GREASE value like 0x5a5a), iterate the list and pick the first
real cipher suite. This is what real TLS servers do per RFC 8701.

Production data shows two client profiles:
- 87% send GREASE first, then 0x1301 (TLS_AES_128_GCM_SHA256)
- 13% send 0xc02b first (TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256)

The fix correctly selects 0x1301 or 0xc02b respectively, matching
real server behavior. Fallback to 0x1301 if all suites are GREASE.

Add snapshot test with GREASE as first cipher suite.
2026-04-06 23:16:08 +03:00
Sergei ArkhipovandGitHub 45f958e527 Merge pull request #441 from appolimp/tcp-keepalive-idle-timeout
Improve TCP keepalive and idle timeout for mobile clients
2026-04-06 21:27:42 +02:00
appolimp 5f81ae3743 Improve TCP keepalive and idle timeout for mobile clients
TCP keepalive was configured (SetKeepAlivePeriod) but never actually
enabled (SO_KEEPALIVE) on accepted client connections. Go 1.26's
SetKeepAlivePeriod only sets TCP_KEEPIDLE — it does not call
setsockopt(SO_KEEPALIVE, 1). Without SO_KEEPALIVE the kernel never
sends probe packets, so dead connections from sleeping mobile clients
linger until the idle timeout fires.

Replace SetKeepAlive + SetKeepAlivePeriod with net.KeepAliveConfig
(available since Go 1.24) for explicit per-socket control:

  Idle:     30s   (time before first probe)
  Interval: 10s   (between probes)
  Count:    3     (failed probes to declare dead)

This detects dead connections in ~60s instead of relying on system
defaults (tcp_keepalive_intvl=75s, probes=9 → up to 11 minutes).

Increase the default idle timeout from 1 minute to 5 minutes.
MTProto clients send ping_delay_disconnect every ~60s, which resets
the idle timer. The previous 1-minute default created a race: if a
ping arrived even 1–2 seconds late the relay was killed. A 5-minute
window also survives typical mobile sleep periods (phone idle 2–5 min)
where the NAT mapping is still alive and the connection can resume
without reconnection.

Ref: #132
2026-04-04 12:01:33 +03:00
9seconds 3a68ea5f2d Update goreleaser 2026-04-01 17:05:30 +02:00
Sergei ArkhipovandGitHub dbced77566 Merge pull request #433 from 9seconds/refactor-tls-fragmentation
Refactor TLS fragmenting
2026-04-01 14:30:21 +02:00
9seconds f4f969e702 Refactor TLS fragmenting 2026-04-01 14:01:24 +02:00
Sergei ArkhipovandGitHub e8368f7645 Merge pull request #431 from appolimp/tls-record-reassembly-pr
Support fragmented TLS handshake records
2026-04-01 09:33:13 +02:00
appolimp 38abee7d7f Support fragmented TLS handshake records
DPI bypass tools like ByeDPI fragment a single TLS record into multiple
records to evade censorship. This broke ReadClientHello because it
assumed the entire ClientHello arrives in one TLS record.

Add reassembleTLSHandshake that reads continuation records and
reconstructs a single TLS record before parsing and HMAC verification.
Per RFC 5246 Section 6.2.1, handshake messages may be fragmented
across multiple records — this is valid TLS behavior.
2026-04-01 09:05:24 +03:00
9seconds a3663fe8b5 Increase timeout for CI artifacts build 2026-03-31 22:13:42 +02:00
9seconds 2aa3321bd4 Add more forks 2026-03-31 19:03:48 +02:00
Sergei ArkhipovandGitHub 3793558c4c Merge pull request #430 from 9seconds/golang-idiomatic
More idiomatic Golang
2026-03-31 17:23:11 +02:00
9seconds b6427ee321 More idiomatic Golang 2026-03-31 15:07:01 +02:00
Sergei ArkhipovandGitHub 0c9fa5e710 Merge pull request #428 from 9seconds/auto-update-prio{
Change IP address set priority
2026-03-31 12:56:58 +02:00
9seconds 1fcec38aea Change IP address set priority
For a couple of releases we use collected IPs as a prioritized source
for connecting to Telegram. But apparently, they work way worse than it
should, and having connectivity to core ip ALWAYS gives better results.
Thus, this PR flips priorities, so users could have auto-update enabled
as a source of secondary addresses, not primary ones
2026-03-31 11:05:49 +02:00
Sergei ArkhipovandGitHub 89930631cf Merge pull request #426 from dolonet/fix/flaky-ci-race-and-bloom 2026-03-31 07:10:00 +02:00
dolonet eedee63143 Address review: use slices.Clone, simplify concurrent test
- Replace manual make+copy with slices.Clone in Snapshot()
- Remove redundant _ = len(data); Snapshot() call alone is
  sufficient to exercise the lock under -race
2026-03-30 16:17:51 +00:00
Alexey Dolotov 73c6a3aa37 fix: tighten ScoutConnCollected encapsulation and add concurrency test
- Move error check before Snapshot() to avoid unnecessary allocation
- Update existing tests to use Snapshot() instead of direct field access
- Add TestConcurrentAddSnapshot to explicitly exercise the mutex
2026-03-30 15:05:50 +03:00
Alexey Dolotov e54d9d60d3 fix: stabilize flaky CI tests
1. Add sync.Mutex to ScoutConnCollected to eliminate data race between
   Add()/MarkWrite() in readLoop and learn() iterating results.
   Introduce Snapshot() for safe read access.

2. Increase bloom filter test size from 500 to 100000 to prevent
   false negatives from random eviction in the stable bloom filter.

3. Use Require().NoError() in TestHTTPSRequest to prevent nil-pointer
   panic on resp.Body.Close() when the request fails.

Fixes #425
2026-03-30 14:50:32 +03:00
9seconds a2de52f071 Update PGO 2026-03-30 13:08:32 +02:00
Sergei ArkhipovandGitHub b926a0590c Merge pull request #424 from dolonet/fix/relay-idle-timeout-shared-tracker
fix: use shared idle tracker for relay connections
2026-03-30 12:54:51 +02:00
Alexey Dolotov 58e8c8f982 ci: trigger tests 2026-03-30 13:51:17 +03:00
Alexey Dolotov 4642546b35 test: add idleTracker and connIdleTimeout tests
Cover shared idle tracker behavior:
- tracker lifecycle (new, idle after timeout, touch resets)
- read/write with data touches tracker
- read retries on timeout when tracker is not idle
- read closes on timeout when tracker is idle
- shared tracker prevents false timeout across directions
2026-03-30 13:21:51 +03:00
Alexey Dolotov 4627910238 fix: use shared idle tracker for relay connections
connIdleTimeout previously set per-direction deadlines independently.
During media downloads the client→telegram direction can be idle at the
application level while telegram→client is actively streaming data.
After IdleTimeout (default 1 min) the idle direction's ReadDeadline
fires, tearing down the entire relay and breaking media transfers.

Replace the per-direction timeout with a shared atomic timestamp that
both pump goroutines update on any successful Read or Write. When a
ReadDeadline fires on the idle direction, we check the shared tracker:
if the other direction was recently active, we retry instead of closing.
The connection is only torn down when both directions are idle for the
full timeout period.

This matches the documented IdleTimeout contract: "if we have any
message which will pass to either direction, a timer is reset."

Overhead: one atomic.Int64 (8 bytes) per connection pair, one
atomic.Store (~1 ns) per Read/Write with data, zero extra goroutines.

Fixes #423
2026-03-30 10:34:16 +03:00
61 changed files with 1787 additions and 214 deletions
+1 -1
View File
@@ -122,7 +122,7 @@ jobs:
artifacts:
name: Build release artifacts
runs-on: ubuntu-latest
timeout-minutes: 10
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v6
+2
View File
@@ -8,3 +8,5 @@
mtg
coverage.txt
dist/
.idea/
.vscode/
+1 -1
View File
@@ -33,7 +33,7 @@ RUN go mod download
COPY . /app
RUN set -x \
&& version="$(git describe --exact-match HEAD 2>/dev/null || git describe --tags --always)" \
&& version="$(git describe --exact-match HEAD 2>/dev/null || git describe --tags --always 2>/dev/null || echo dev)" \
&& go build \
-trimpath \
-mod=readonly \
+46
View File
@@ -29,6 +29,8 @@ are the most notable:
* [Official](https://github.com/TelegramMessenger/MTProxy)
* [Python](https://github.com/alexbers/mtprotoproxy)
* [Erlang](https://github.com/seriyps/mtproto_proxy)
* [Teleproxy (C)](https://github.com/teleproxy/teleproxy)
* [mtproto.zig (Zig)](https://github.com/sleep3r/mtproto.zig)
* [Telemt (Rust)](https://github.com/telemt/telemt)
You can use any of these. They work great and all implementations have
@@ -512,6 +514,50 @@ This is not very necessary. Keep in mind these rules:
you can enable `drs` setting.
9. **If you are not sure, touch nothing!**
## Troubleshooting
### `ip was blacklisted` for clients on the same LAN
If you run mtg at home and a client on the same LAN (for example, your
phone on the home Wi-Fi) cannot connect, check the proxy logs for a
message like:
```json
{"level":"info","ip":"10.0.1.1","logger":"proxy","message":"ip was blacklisted"}
```
The reason is that the default blocklist (`firehol_level1.netset`)
includes bogon networks, which covers all RFC1918 ranges
(`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`). Any client
connecting from such an address is rejected by the blocklist —
the TCP connection is closed immediately with no response, so
from the client's point of view nothing loads at all.
There are three ways to resolve it:
1. Disable the blocklist entirely in `config.toml`:
```toml
[defense.blocklist]
enabled = false
```
Simplest option if the proxy is used only by you and people you trust.
2. Keep the blocklist but swap `firehol_level1` for a narrower list that
does not include bogons, for example `firehol_abusers_1d`:
```toml
[defense.blocklist]
enabled = true
urls = ["https://iplists.firehol.org/files/firehol_abusers_1d.netset"]
```
3. Connect to the proxy through a public IP or domain name with hairpin
NAT (`MASQUERADE`) on your router. mtg will then see the client with
its public address and the blocklist will not match. This is more
work to set up but preserves full blocklist protection.
## Metrics
Out of the box, mtg works with
+1 -1
View File
@@ -12,7 +12,7 @@ type StableBloomFilterTestSuite struct {
}
func (suite *StableBloomFilterTestSuite) TestOp() {
filter := antireplay.NewStableBloomFilter(500, 0.001)
filter := antireplay.NewStableBloomFilter(100000, 0.001)
suite.False(filter.SeenBefore([]byte{1, 2, 3}))
suite.False(filter.SeenBefore([]byte{4, 5, 6}))
+30
View File
@@ -0,0 +1,30 @@
{
# Caddy sits behind HAProxy which passes raw TLS through on :8443.
# ACME HTTP-01 challenges arrive on :80 via HAProxy's acl passthrough.
http_port 80
https_port 8443
# HAProxy forwards connections to :8443 with a PROXY protocol v2
# header (see haproxy.cfg `send-proxy-v2`). The proxy_protocol
# listener wrapper strips the header and exposes the real client IP
# to Caddy's access log. The `tls` wrapper must follow so that TLS
# is terminated on the unwrapped connection.
#
# `allow` lists the networks permitted to send PROXY headers. These
# ranges cover docker compose's default bridge networks; tighten
# them if you pin a specific subnet in docker-compose.yml.
servers :8443 {
listener_wrappers {
proxy_protocol {
timeout 5s
allow 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
}
tls
}
}
}
{$DOMAIN} {
root * /srv
file_server
}
+89
View File
@@ -0,0 +1,89 @@
# SNI-routing deployment for mtg
A turnkey `docker compose` setup that puts an SNI-aware TCP router
(HAProxy) in front of mtg **and** a real web server (Caddy with
automatic HTTPS).
## Why
Modern DPI systems actively probe suspected proxies. If the server
closes the connection or returns something unexpected, the IP gets
flagged. With this setup:
- **Telegram clients** connect to port 443, HAProxy sees the configured
SNI and routes them to mtg (FakeTLS).
- **Everything else** (browsers, DPI probes, scanners) is routed to
Caddy, which responds with a real Let's Encrypt certificate and serves
genuine web content.
Because your domain's DNS points to this server, the SNI/IP match is
natural and passive DPI has nothing to flag.
## Quick start
```bash
# 1. Point your domain's DNS A/AAAA record to this server's IP.
# 2. Generate an mtg secret:
docker run --rm nineseconds/mtg:2 generate-secret --hex YOUR_DOMAIN
# 3. Edit the config files:
# - mtg-config.toml → paste the secret
# - haproxy.cfg → replace "example.com" in the SNI ACL
# - .env or export → DOMAIN=your.domain
# 4. (Optional) put your site content into www/
# 5. Start:
docker compose up -d
# 6. Verify:
# - Open https://YOUR_DOMAIN in a browser → you should see the web page
# - Configure Telegram with the proxy link from:
docker compose exec mtg mtg access /config/config.toml
```
## Real client IPs (PROXY protocol)
HAProxy forwards TCP connections to mtg and Caddy with a PROXY protocol
v2 header so both backends see the real client IP instead of HAProxy's
container address. The three pieces must stay in sync:
- `haproxy.cfg``send-proxy-v2` on the `mtg` and `web` backend `server` lines
- `mtg-config.toml``proxy-protocol-listener = true`
- `Caddyfile``listener_wrappers { proxy_protocol { ... } tls }` on `:8443`
If you disable one, disable all three, otherwise the backend will fail
to parse the connection.
## ACME (Let's Encrypt) notes
HAProxy passes `/.well-known/acme-challenge/` requests on `:80` to
Caddy so that HTTP-01 validation works out of the box. Make sure your
domain's DNS A/AAAA record points to this server before starting.
## Architecture
```
┌──────────────────┐
:443 ──────>│ HAProxy │
│ (TCP, SNI peek) │
└──┬───────────┬───┘
SNI match │ │ default
v v
┌─────────┐ ┌─────────┐
│ mtg │ │ Caddy │
│ :3128 │ │ :8443 │
│ FakeTLS │ │ real TLS│
└─────────┘ └─────────┘
```
## Files
| File | Purpose |
|---|---|
| `docker-compose.yml` | Service definitions |
| `haproxy.cfg` | SNI routing rules — **edit the domain** |
| `mtg-config.toml` | mtg proxy config — **paste your secret** |
| `Caddyfile` | Web server config (auto-HTTPS) |
| `www/` | Static site content served by Caddy |
+54
View File
@@ -0,0 +1,54 @@
# SNI-routing deployment: HAProxy (443) -> mtg + real web backend
#
# This setup puts an SNI-aware TCP router in front of mtg so that:
# - Telegram clients (FakeTLS with the correct SNI) are routed to mtg
# - All other TLS traffic (including DPI probes) reaches the real web
# server, which responds with a genuine certificate
#
# The result: active probes see a real website; passive DPI sees matching
# SNI/IP because the domain resolves to this server's IP.
#
# Quick start:
# 1. Set YOUR_DOMAIN below (and in mtg-config.toml)
# 2. docker compose up -d
# 3. mtg generate-secret YOUR_DOMAIN -> put it in mtg-config.toml
# 4. docker compose restart mtg
#
# See BEST_PRACTICES.md and the project wiki for background.
services:
haproxy:
image: haproxy:lts-alpine
ports:
- "443:443"
- "80:80"
volumes:
- ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
depends_on:
- mtg
- web
restart: unless-stopped
mtg:
image: nineseconds/mtg:2
volumes:
- ./mtg-config.toml:/config/config.toml:ro
expose:
- "3128"
restart: unless-stopped
web:
image: caddy:alpine
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data
- ./www:/srv:ro
expose:
- "80"
- "8443"
environment:
DOMAIN: ${DOMAIN:-example.com}
restart: unless-stopped
volumes:
caddy_data:
+62
View File
@@ -0,0 +1,62 @@
# HAProxy SNI router — Layer 4 (TCP mode)
#
# Inspects the SNI in the TLS ClientHello and routes traffic:
# - SNI matching the mtg secret domain -> mtg (FakeTLS / MTProto)
# - Everything else -> real web backend (Caddy)
#
# Because routing happens before TLS termination, each backend sees the
# raw ClientHello and handles TLS itself. The real web backend therefore
# presents a genuine certificate to any probe or browser.
global
log stdout format raw local0 info
maxconn 4096
defaults
log global
mode tcp
option tcplog
timeout connect 5s
timeout client 60s
timeout server 60s
# --- HTTP :80 — ACME challenges + redirect -----------------------------------
frontend http
bind *:80
mode http
# Let Caddy answer ACME HTTP-01 challenges for Let's Encrypt.
acl is_acme path_beg /.well-known/acme-challenge/
use_backend web_acme if is_acme
http-request redirect scheme https code 301
# --- TLS :443 — SNI-based routing -------------------------------------------
frontend tls
bind *:443
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
# Route Telegram clients to mtg.
# Replace "example.com" with the domain from your mtg secret.
use_backend mtg if { req_ssl_sni -i example.com }
default_backend web
backend mtg
# send-proxy-v2 prepends a PROXY protocol v2 header so mtg sees the
# real client IP instead of HAProxy's. mtg must have
# `proxy-protocol-listener = true` in its config.
server mtg mtg:3128 send-proxy-v2
backend web
# send-proxy-v2 prepends a PROXY protocol v2 header so Caddy logs the
# real client IP instead of HAProxy's. Caddy must enable the
# proxy_protocol listener wrapper on :8443 (see Caddyfile).
server web web:8443 send-proxy-v2
backend web_acme
mode http
server web web:80
+17
View File
@@ -0,0 +1,17 @@
# Minimal mtg configuration for the SNI-router setup.
#
# 1. Generate a secret: mtg generate-secret --hex example.com
# 2. Paste it below.
# 3. Replace example.com with your actual domain everywhere.
secret = "PASTE_YOUR_SECRET_HERE"
bind-to = "0.0.0.0:3128"
# HAProxy in front sends PROXY protocol v2 headers so mtg can see the
# real client IP. Keep this in sync with haproxy.cfg (`send-proxy-v2`).
proxy-protocol-listener = true
[defense.anti-replay]
enabled = true
max-size = "1mib"
error-rate = 0.001
+5
View File
@@ -0,0 +1,5 @@
<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>Welcome</title></head>
<body><h1>It works!</h1><p>Replace this with your own content.</p></body>
</html>
BIN
View File
Binary file not shown.
+13 -1
View File
@@ -25,6 +25,18 @@ var TelegramCoreAddresses = map[int][]string{
},
203: {
"91.105.192.100:443",
"[2a0a:f280:0203:000a:5000:0000:0000:0100]:443",
"[2a0a:f280:0203:a:5000::100]:443",
},
10001: {
"149.154.175.10:443",
"[2001:b28:f23d:f001::e]:443",
},
10002: {
"149.154.167.40:443",
"[2001:67c:4e8:f002::e]:443",
},
10003: {
"149.154.175.117:443",
"[2001:b28:f23d:f003::e]:443",
},
}
+32 -5
View File
@@ -204,14 +204,22 @@ proxies = [
# define a global timeout on establishing of network connections. idle
# means a timeout on pumping data between sockset when nothing is
# happening.
#
# please be noticed that handshakes have no timeouts intentionally. You can
# find a reasoning here:
# https://www.ndss-symposium.org/wp-content/uploads/2020/02/23087-paper.pdf
[network.timeout]
tcp = "5s"
http = "10s"
idle = "1m"
idle = "5m"
handshake = "10s"
# this defines a configuration for TCP keep alives. Default values are taken
# from Golang default behavior.
[network.keep-alive]
disabled = false
# idle means a time period after which we start sending TCP Keep Alive probes
idle = "15s"
# interval is a period between 2 consecutive probes
interval = "15s"
# if we miss that many probes, a connection will be considered as a dead one.
count = 9
# mtg has to mimic real websites. It does not mean domain fronting, it also
# means that traffic characteristics should be similar to real world traffic.
@@ -308,6 +316,17 @@ download-concurrency = 2
# A list of URLs in FireHOL format (https://iplists.firehol.org/)
# You can provider links here (starts with https:// or http://) or
# path to a local file, but in this case it should be absolute.
#
# NOTE: the default list below (firehol_level1.netset) includes bogon
# networks, and therefore RFC1918 ranges as well (10.0.0.0/8,
# 172.16.0.0/12, 192.168.0.0/16). If you run mtg on a home/LAN network
# and connect from a client on the same LAN, that client will be
# rejected with "ip was blacklisted" and the connection dropped (TCP
# close, no response). If you see this, you can either disable this section
# (enabled = false), replace firehol_level1 with a narrower list that
# does not include bogons (e.g. firehol_abusers_1d), or connect via
# a public IP/domain with hairpin NAT on your router. See README for
# details.
urls = [
"https://iplists.firehol.org/files/firehol_level1.netset",
# "/local.file"
@@ -359,3 +378,11 @@ bind-to = "127.0.0.1:3129"
http-path = "/"
# prefix for metrics for prometheus
metric-prefix = "mtg"
[dc]
dc1 = [
"149.154.175.50:443",
"149.154.175.54:443",
"149.154.175.59:443",
"[2001:b28:f23d:f001::a]:443"
]
+2 -2
View File
@@ -4,7 +4,7 @@ go 1.26
require (
github.com/OneOfOne/xxhash v1.2.8
github.com/alecthomas/kong v1.14.0
github.com/alecthomas/kong v1.15.0
github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5
github.com/babolivier/go-doh-client v0.0.0-20201028162107-a76cff4cb8b6
@@ -28,7 +28,7 @@ require (
require (
github.com/beevik/ntp v1.5.0
github.com/ncruces/go-dns v1.3.2
github.com/ncruces/go-dns v1.3.3
github.com/pelletier/go-toml/v2 v2.3.0
github.com/pires/go-proxyproto v0.11.0
github.com/things-go/go-socks5 v0.1.0
+4 -4
View File
@@ -2,8 +2,8 @@ github.com/OneOfOne/xxhash v1.2.8 h1:31czK/TI9sNkxIKfaUfGlU47BAxQ0ztGgd9vPyqimf8
github.com/OneOfOne/xxhash v1.2.8/go.mod h1:eZbhyaAYD41SGSSsnmcpxVoRiQ/MPUTjUdIIOT9Um7Q=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/kong v1.14.0 h1:gFgEUZWu2ZmZ+UhyZ1bDhuutbKN1nTtJTwh19Wsn21s=
github.com/alecthomas/kong v1.14.0/go.mod h1:wrlbXem1CWqUV5Vbmss5ISYhsVPkBb1Yo7YKJghju2I=
github.com/alecthomas/kong v1.15.0 h1:BVJstKbpO73zKpmIu+m/aLRrNmWwxXPIGTNin9VmLVI=
github.com/alecthomas/kong v1.15.0/go.mod h1:wrlbXem1CWqUV5Vbmss5ISYhsVPkBb1Yo7YKJghju2I=
github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vSQ6PWWSL9lK8qwHozUj03+zLoEB8O0=
@@ -48,8 +48,8 @@ github.com/miekg/dns v1.1.51 h1:0+Xg7vObnhrz/4ZCZcZh7zPXlmU0aveS2HDBd0m0qSo=
github.com/miekg/dns v1.1.51/go.mod h1:2Z9d3CP1LQWihRZUf29mQ19yDThaI4DAYzte2CaQW5c=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/ncruces/go-dns v1.3.2 h1:kBLuUZBgkQ4qF4WDXZRQ4rG0Gk6sLVJQ5tESkWrxUa0=
github.com/ncruces/go-dns v1.3.2/go.mod h1:tuzixNY8PY/M7yUzcvRbUaeLs3ifIdydpi5H2bfRU+s=
github.com/ncruces/go-dns v1.3.3 h1:59OV7XoJrTCoUMZjWRVs4GOjtntMTZqiQ5Mn+BT13hk=
github.com/ncruces/go-dns v1.3.3/go.mod h1:tuzixNY8PY/M7yUzcvRbUaeLs3ifIdydpi5H2bfRU+s=
github.com/panjf2000/ants/v2 v2.12.0 h1:u9JhESo83i/GkZnhfTNuFMMWcNt7mnV1bGJ6FT4wXH8=
github.com/panjf2000/ants/v2 v2.12.0/go.mod h1:tSQuaNQ6r6NRhPt+IZVUevvDyFMTs+eS4ztZc52uJTY=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
+19
View File
@@ -50,6 +50,12 @@ var (
tplEDCConnect = template.Must(
template.New("").Parse(" ❌ DC {{ .dc }}: {{ .error }}\n"),
)
tplODCIPConnect = template.Must(
template.New("").Parse(" ✅ {{ .ip }}\n"),
)
tplEDCIPConnect = template.Must(
template.New("").Parse(" ❌ {{ .ip }}: {{ .error }}\n"),
)
tplODNSSNIMatch = template.Must(
template.New("").Parse(" ✅ IP address {{ .ip }} matches secret hostname {{ .hostname }}\n"),
@@ -97,6 +103,12 @@ func (d *Doctor) Run(cli *CLI, version string) error {
conf.Network.Timeout.TCP.Get(10*time.Second),
conf.Network.Timeout.HTTP.Get(0),
conf.Network.Timeout.Idle.Get(0),
net.KeepAliveConfig{
Enable: !conf.Network.KeepAlive.Disabled.Get(false),
Idle: conf.Network.KeepAlive.Idle.Get(0),
Interval: conf.Network.KeepAlive.Interval.Get(0),
Count: int(conf.Network.KeepAlive.Count.Get(0)),
},
)
fmt.Println("Validate native network connectivity")
@@ -279,8 +291,15 @@ func (d *Doctor) checkNetworkAddresses(ntw mtglib.Network, addresses []string) e
for _, addr := range checkAddresses {
conn, err = ntw.DialContext(ctx, "tcp", addr)
if err != nil {
tplEDCIPConnect.Execute(os.Stdout, map[string]any{
"error": err,
"ip": addr,
})
continue
}
tplODCIPConnect.Execute(os.Stdout, map[string]any{
"ip": addr,
})
conn.Close() //nolint: errcheck
+8 -2
View File
@@ -5,7 +5,6 @@ import (
"fmt"
"net"
"os"
"time"
"github.com/9seconds/mtg/v2/antireplay"
"github.com/9seconds/mtg/v2/events"
@@ -51,6 +50,12 @@ func makeNetwork(conf *config.Config, version string) (mtglib.Network, error) {
conf.Network.Timeout.TCP.Get(0),
conf.Network.Timeout.HTTP.Get(0),
conf.Network.Timeout.Idle.Get(0),
net.KeepAliveConfig{
Enable: !conf.Network.KeepAlive.Disabled.Get(false),
Idle: conf.Network.KeepAlive.Idle.Get(0),
Interval: conf.Network.KeepAlive.Interval.Get(0),
Count: int(conf.Network.KeepAlive.Count.Get(0)),
},
)
proxyDialers := make([]mtglib.Network, len(conf.Network.Proxies))
@@ -263,7 +268,8 @@ func runProxy(conf *config.Config, version string) error { //nolint: funlen
AllowFallbackOnUnknownDC: conf.AllowFallbackOnUnknownDC.Get(false),
TolerateTimeSkewness: conf.TolerateTimeSkewness.Value,
IdleTimeout: conf.Network.Timeout.Idle.Get(time.Minute),
IdleTimeout: conf.Network.Timeout.Idle.Get(mtglib.DefaultIdleTimeout),
HandshakeTimeout: conf.Network.Timeout.Handshake.Get(mtglib.DefaultHandshakeTimeout),
DoppelGangerURLs: doppelGangerURLs,
DoppelGangerPerRaid: conf.Defense.Doppelganger.Repeats.Get(mtglib.DoppelGangerPerRaid),
+18
View File
@@ -63,7 +63,14 @@ type Config struct {
TCP TypeDuration `json:"tcp"`
HTTP TypeDuration `json:"http"`
Idle TypeDuration `json:"idle"`
Handshake TypeDuration `json:"handshake"`
} `json:"timeout"`
KeepAlive struct {
Disabled TypeBool `json:"disabled"`
Idle TypeDuration `json:"idle"`
Interval TypeDuration `json:"interval"`
Count TypeConcurrency `json:"count"`
} `json:"keepAlive"`
DOHIP TypeIP `json:"dohIp"`
DNS TypeDNSURI `json:"dns"`
Proxies []TypeProxyURL `json:"proxies"`
@@ -84,6 +91,17 @@ type Config struct {
MetricPrefix TypeMetricPrefix `json:"metricPrefix"`
} `json:"prometheus"`
} `json:"stats"`
DCs struct{
DC1 []TypeHostPort `json:"dc1"`
DC2 []TypeHostPort `json:"dc2"`
DC3 []TypeHostPort `json:"dc3"`
DC4 []TypeHostPort `json:"dc4"`
DC5 []TypeHostPort `json:"dc5"`
DC203 []TypeHostPort `json:"dc203"`
DC10001 []TypeHostPort `json:"dc10001"`
DC10002[]TypeHostPort `json:"dc10002"`
DC10003 []TypeHostPort `json:"dc10003"`
} `json:"dc"`
}
func (c *Config) GetConcurrency(defaultValue uint) uint {
+18
View File
@@ -58,7 +58,14 @@ type tomlConfig struct {
TCP string `toml:"tcp" json:"tcp,omitempty"`
HTTP string `toml:"http" json:"http,omitempty"`
Idle string `toml:"idle" json:"idle,omitempty"`
Handshake string `toml:"handshake" json:"handshake,omitempty"`
} `toml:"timeout" json:"timeout,omitempty"`
KeepAlive struct {
Disabled bool `toml:"disabled" json:"disabled,omitempty"`
Idle string `toml:"idle" json:"idle,omitempty"`
Interval string `toml:"interval" json:"interval,omitempty"`
Count uint `toml:"count" json:"count,omitempty"`
} `toml:"keep-alive" json:"keepAlive,omitempty"`
DOHIP string `toml:"doh-ip" json:"dohIp,omitempty"`
DNS string `toml:"dns" json:"dns,omitempty"`
Proxies []string `toml:"proxies" json:"proxies,omitempty"`
@@ -77,6 +84,17 @@ type tomlConfig struct {
MetricPrefix string `toml:"metric-prefix" json:"metricPrefix,omitempty"`
} `toml:"prometheus" json:"prometheus,omitempty"`
} `toml:"stats" json:"stats,omitempty"`
DCs struct{
DC1 []string `toml:"dc1" json:"dc1,omitempty"`
DC2 []string `toml:"dc2" json:"dc2,omitempty"`
DC3 []string `toml:"dc3" json:"dc3,omitempty"`
DC4 []string `toml:"dc4" json:"dc4,omitempty"`
DC5 []string `toml:"dc5" json:"dc5,omitempty"`
DC203 []string `toml:"dc203" json:"dc203,omitempty"`
DC10001 []string `toml:"dc10001" json:"dc10001,omitempty"`
DC10002 []string `toml:"dc10002" json:"dc10002,omitempty"`
DC10003 []string `toml:"dc10003" json:"dc10003,omitempty"`
} `toml:"dc" json:"dc,omitempty"`
}
func Parse(rawData []byte) (*Config, error) {
+70
View File
@@ -4,9 +4,77 @@ import (
"fmt"
"os"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/internal/config"
)
func readCustomDC(cfg *config.Config) {
if len(cfg.DCs.DC1) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC1 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[1] = ips
}
if len(cfg.DCs.DC2) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC2 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[2] = ips
}
if len(cfg.DCs.DC3) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC3 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[3] = ips
}
if len(cfg.DCs.DC4) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC4 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[4] = ips
}
if len(cfg.DCs.DC5) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC5 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[5] = ips
}
if len(cfg.DCs.DC203) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC203 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[203] = ips
}
if len(cfg.DCs.DC10001) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC10001 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[10001] = ips
}
if len(cfg.DCs.DC10002) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC10002 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[10002] = ips
}
if len(cfg.DCs.DC10003) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC10003 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[10003] = ips
}
}
func ReadConfig(path string) (*config.Config, error) {
content, err := os.ReadFile(path)
if err != nil {
@@ -22,5 +90,7 @@ func ReadConfig(path string) (*config.Config, error) {
return nil, fmt.Errorf("invalid config: %w", err)
}
readCustomDC(conf)
return conf, nil
}
+16 -16
View File
@@ -82,40 +82,40 @@ checksum = "sha256:4932cfca5e75bf60fe1c576edf459e5e809e6644664a068185d64b84af3fa
url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-windows-amd64.zip"
[[tools.goreleaser]]
version = "2.14.3"
version = "2.15.2"
backend = "aqua:goreleaser/goreleaser"
[tools.goreleaser."platforms.linux-arm64"]
checksum = "sha256:581a10e53c1176b3e81ee45cf531e02dbf899db0bc7b795669347df4276ce948"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Linux_arm64.tar.gz"
checksum = "sha256:5db66761a98f6693161e49e1a95d28d2673a892ba60cb4a5e16736cafd41c4c9"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_arm64.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.linux-arm64-musl"]
checksum = "sha256:581a10e53c1176b3e81ee45cf531e02dbf899db0bc7b795669347df4276ce948"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Linux_arm64.tar.gz"
checksum = "sha256:5db66761a98f6693161e49e1a95d28d2673a892ba60cb4a5e16736cafd41c4c9"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_arm64.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.linux-x64"]
checksum = "sha256:dc7faeeeb6da8bdfda788626263a4ae725892a8c7504b975c3234127d4a44579"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Linux_x86_64.tar.gz"
checksum = "sha256:0ebdbf0353aba566b969dde746cc4e4806f96c27aa2f3971b229a9df7611fedc"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_x86_64.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.linux-x64-musl"]
checksum = "sha256:dc7faeeeb6da8bdfda788626263a4ae725892a8c7504b975c3234127d4a44579"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Linux_x86_64.tar.gz"
checksum = "sha256:0ebdbf0353aba566b969dde746cc4e4806f96c27aa2f3971b229a9df7611fedc"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_x86_64.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.macos-arm64"]
checksum = "sha256:3507798489e107a78aff36b169de48148a335ac26eb3161608d905f3f3a957bd"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Darwin_all.tar.gz"
provenance = "cosign"
checksum = "sha256:0e6bd67688ac949780bf1166813a91f89856898ef4c40d7d46c2c74ebaa4b9ee"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Darwin_all.tar.gz"
provenance = "github-attestations"
[tools.goreleaser."platforms.macos-x64"]
checksum = "sha256:3507798489e107a78aff36b169de48148a335ac26eb3161608d905f3f3a957bd"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Darwin_all.tar.gz"
checksum = "sha256:0e6bd67688ac949780bf1166813a91f89856898ef4c40d7d46c2c74ebaa4b9ee"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Darwin_all.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.windows-x64"]
checksum = "sha256:3deea8ff471aa258a2d99f3e5302971d7028647ae8ddaf103257a8113e485a31"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Windows_x86_64.zip"
checksum = "sha256:7459832946dbe122c144f8d7f87484d8572ca005b779310aa6bb03346e8de17a"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Windows_x86_64.zip"
provenance = "cosign"
+53 -8
View File
@@ -3,9 +3,11 @@ package mtglib
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"net"
"sync/atomic"
"time"
"github.com/9seconds/mtg/v2/essentials"
@@ -97,20 +99,63 @@ func newConnProxyProtocol(source, target essentials.Conn) *connProxyProtocol {
}
}
type connIdleTimeout struct {
essentials.Conn
// idleTracker is a shared idle tracker for a pair of relay connections.
// Both directions update the same timestamp so that activity in one direction
// prevents the other (idle) direction from timing out.
type idleTracker struct {
lastActive atomic.Pointer[time.Time]
timeout time.Duration
}
func (c connIdleTimeout) Read(b []byte) (int, error) {
c.SetReadDeadline(time.Now().Add(c.timeout)) //nolint: errcheck
func newIdleTracker(timeout time.Duration) *idleTracker {
t := &idleTracker{timeout: timeout}
t.touch()
return c.Conn.Read(b) //nolint: wrapcheck
return t
}
func (t *idleTracker) touch() {
stamp := time.Now()
t.lastActive.Store(&stamp)
}
func (t *idleTracker) isIdle() bool {
return time.Since(*t.lastActive.Load()) >= t.timeout
}
type connIdleTimeout struct {
essentials.Conn
tracker *idleTracker
}
func (c connIdleTimeout) Read(b []byte) (int, error) {
var netErr net.Error
for {
c.SetReadDeadline(time.Now().Add(c.tracker.timeout)) //nolint: errcheck
n, err := c.Conn.Read(b)
switch {
case err == nil:
c.tracker.touch()
return n, nil
case errors.As(err, &netErr) && netErr.Timeout() && !c.tracker.isIdle():
continue
}
return n, err
}
}
func (c connIdleTimeout) Write(b []byte) (int, error) {
c.SetWriteDeadline(time.Now().Add(c.timeout)) //nolint: errcheck
c.SetWriteDeadline(time.Now().Add(c.tracker.timeout)) //nolint: errcheck
return c.Conn.Write(b) //nolint: wrapcheck
n, err := c.Conn.Write(b)
if n > 0 {
c.tracker.touch()
}
return n, err //nolint: wrapcheck
}
+151
View File
@@ -16,6 +16,12 @@ import (
"github.com/stretchr/testify/suite"
)
type netTimeoutError struct{}
func (e netTimeoutError) Error() string { return "i/o timeout" }
func (e netTimeoutError) Timeout() bool { return true }
func (e netTimeoutError) Temporary() bool { return true }
type ConnRewindBaseConn struct {
testlib.EssentialsConnMock
@@ -291,6 +297,141 @@ func (suite *ConnProxyProtocolTestSuite) TearDownTest() {
suite.targetConnMock.AssertExpectations(suite.T())
}
type IdleTrackerTestSuite struct {
suite.Suite
}
func (suite *IdleTrackerTestSuite) TestNewNotIdle() {
tracker := newIdleTracker(time.Second)
suite.False(tracker.isIdle())
}
func (suite *IdleTrackerTestSuite) TestIdleAfterTimeout() {
tracker := newIdleTracker(10 * time.Millisecond)
time.Sleep(20 * time.Millisecond)
suite.True(tracker.isIdle())
}
func (suite *IdleTrackerTestSuite) TestTouchResetsIdle() {
tracker := newIdleTracker(50 * time.Millisecond)
time.Sleep(30 * time.Millisecond)
tracker.touch()
suite.False(tracker.isIdle())
}
type ConnIdleTimeoutTestSuite struct {
suite.Suite
connMock *testlib.EssentialsConnMock
tracker *idleTracker
conn connIdleTimeout
}
func (suite *ConnIdleTimeoutTestSuite) SetupTest() {
suite.connMock = &testlib.EssentialsConnMock{}
suite.tracker = newIdleTracker(time.Second)
suite.conn = connIdleTimeout{
Conn: suite.connMock,
tracker: suite.tracker,
}
}
func (suite *ConnIdleTimeoutTestSuite) TearDownTest() {
suite.connMock.AssertExpectations(suite.T())
}
func (suite *ConnIdleTimeoutTestSuite) TestReadOk() {
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(5, nil)
n, err := suite.conn.Read(make([]byte, 10))
suite.NoError(err)
suite.Equal(5, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestReadNonTimeoutErr() {
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, io.EOF)
n, err := suite.conn.Read(make([]byte, 10))
suite.True(errors.Is(err, io.EOF))
suite.Equal(0, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestReadTimeoutRetriesWhenNotIdle() {
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, netTimeoutError{})
suite.connMock.On("Read", mock.Anything).Once().Return(5, nil)
n, err := suite.conn.Read(make([]byte, 10))
suite.NoError(err)
suite.Equal(5, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestReadTimeoutClosesWhenIdle() {
suite.tracker = newIdleTracker(time.Millisecond)
suite.conn = connIdleTimeout{
Conn: suite.connMock,
tracker: suite.tracker,
}
time.Sleep(5 * time.Millisecond)
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, netTimeoutError{})
n, err := suite.conn.Read(make([]byte, 10))
suite.Equal(0, n)
netErr, ok := err.(net.Error) //nolint: errorlint
suite.True(ok)
suite.True(netErr.Timeout())
}
func (suite *ConnIdleTimeoutTestSuite) TestSharedTrackerPreventsFalseTimeout() {
connMock2 := &testlib.EssentialsConnMock{}
conn2 := connIdleTimeout{
Conn: connMock2,
tracker: suite.tracker,
}
connMock2.On("SetWriteDeadline", mock.Anything).Return(nil)
connMock2.On("Write", mock.Anything).Once().Return(5, nil)
_, _ = conn2.Write(make([]byte, 5))
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, netTimeoutError{})
suite.connMock.On("Read", mock.Anything).Once().Return(3, nil)
n, err := suite.conn.Read(make([]byte, 10))
suite.NoError(err)
suite.Equal(3, n)
connMock2.AssertExpectations(suite.T())
}
func (suite *ConnIdleTimeoutTestSuite) TestWriteOk() {
suite.connMock.On("SetWriteDeadline", mock.Anything).Return(nil)
suite.connMock.On("Write", mock.Anything).Once().Return(5, nil)
n, err := suite.conn.Write(make([]byte, 5))
suite.NoError(err)
suite.Equal(5, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestWriteErr() {
suite.connMock.On("SetWriteDeadline", mock.Anything).Return(nil)
suite.connMock.On("Write", mock.Anything).Once().Return(0, io.EOF)
n, err := suite.conn.Write(make([]byte, 5))
suite.True(errors.Is(err, io.EOF))
suite.Equal(0, n)
}
func TestConnTraffic(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnTrafficTestSuite{})
@@ -305,3 +446,13 @@ func TestConnProxyProtocol(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnProxyProtocolTestSuite{})
}
func TestIdleTracker(t *testing.T) {
t.Parallel()
suite.Run(t, &IdleTrackerTestSuite{})
}
func TestConnIdleTimeout(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnIdleTimeoutTestSuite{})
}
+7 -2
View File
@@ -77,8 +77,13 @@ const (
// DefaultIdleTimeout is a default timeout for closing a connection in case of
// idling.
//
// Deprecated: no longer in use because of changed TCP relay algorithm.
DefaultIdleTimeout = time.Minute
// Set to 5 minutes to survive typical mobile sleep periods (2-5 min) and
// avoid racing with MTProto ping_delay_disconnect (~60s interval).
DefaultIdleTimeout = 5 * time.Minute
// DefaultHandshakeTimeout defines a time period during which the
// all handshake ceremonies must be completed.
DefaultHandshakeTimeout = 10 * time.Second
// DefaultTolerateTimeSkewness is a default timeout for time skewness on a
// faketls timeout verification.
+6 -2
View File
@@ -5,15 +5,19 @@ type dcView struct {
}
func (d dcView) getV4(dc int) []Addr {
addrs := d.publicConfigs.getV4(dc)
var addrs []Addr
addrs = append(addrs, defaultDCAddrSet.getV4(dc)...)
addrs = append(addrs, d.publicConfigs.getV4(dc)...)
return addrs
}
func (d dcView) getV6(dc int) []Addr {
addrs := d.publicConfigs.getV6(dc)
var addrs []Addr
addrs = append(addrs, defaultDCAddrSet.getV6(dc)...)
addrs = append(addrs, d.publicConfigs.getV6(dc)...)
return addrs
}
+12 -6
View File
@@ -61,23 +61,29 @@ func (s Scout) learn(ctx context.Context, url string) (ScoutResult, error) {
client.CloseIdleConnections()
}
if err != nil || len(results.data) == 0 {
if err != nil {
return ScoutResult{}, err
}
data, writeIndex := results.Snapshot()
if len(data) == 0 {
return ScoutResult{}, nil
}
var result ScoutResult
// Compute inter-record durations (existing logic).
lastTimestamp := time.Time{}
for i, v := range results.data {
for i, v := range data {
if v.recordType != tls.TypeApplicationData {
continue
}
if lastTimestamp.IsZero() {
if i > 0 {
lastTimestamp = results.data[i-1].timestamp
lastTimestamp = data[i-1].timestamp
} else {
lastTimestamp = v.timestamp
}
@@ -90,12 +96,12 @@ func (s Scout) learn(ctx context.Context, url string) (ScoutResult, error) {
// Compute cert size: sum of ApplicationData payload between CCS and
// the first client Write (which marks the end of server handshake).
seenCCS := false
boundary := results.writeIndex
boundary := writeIndex
if boundary < 0 {
boundary = len(results.data)
boundary = len(data)
}
for i, v := range results.data {
for i, v := range data {
if i >= boundary {
break
}
+20 -1
View File
@@ -1,6 +1,10 @@
package doppel
import "time"
import (
"slices"
"sync"
"time"
)
const (
ScoutConnCollectedPreallocSize = 100
@@ -13,23 +17,38 @@ type ScoutConnResult struct {
}
type ScoutConnCollected struct {
mu sync.Mutex
data []ScoutConnResult
writeIndex int // index at which client first wrote post-handshake data; -1 if not set
}
func (s *ScoutConnCollected) Add(record byte, payloadLen int) {
s.mu.Lock()
s.data = append(s.data, ScoutConnResult{
timestamp: time.Now(),
recordType: record,
payloadLen: payloadLen,
})
s.mu.Unlock()
}
// MarkWrite records the current data length as the handshake boundary.
func (s *ScoutConnCollected) MarkWrite() {
s.mu.Lock()
if s.writeIndex < 0 {
s.writeIndex = len(s.data)
}
s.mu.Unlock()
}
// Snapshot returns a copy of the collected data and the write index.
func (s *ScoutConnCollected) Snapshot() ([]ScoutConnResult, int) {
s.mu.Lock()
snapshot := slices.Clone(s.data)
writeIndex := s.writeIndex
s.mu.Unlock()
return snapshot, writeIndex
}
func NewScoutConnCollected() *ScoutConnCollected {
@@ -1,6 +1,7 @@
package doppel
import (
"sync"
"testing"
"time"
@@ -16,8 +17,10 @@ func (suite *ScoutConnCollectedTestSuite) TestAddSingle() {
collected := NewScoutConnCollected()
collected.Add(tls.TypeApplicationData, 100)
suite.Len(collected.data, 1)
suite.Equal(byte(tls.TypeApplicationData), collected.data[0].recordType)
data, _ := collected.Snapshot()
suite.Len(data, 1)
suite.Equal(byte(tls.TypeApplicationData), data[0].recordType)
}
func (suite *ScoutConnCollectedTestSuite) TestAddTimestampsAreMonotonic() {
@@ -31,11 +34,52 @@ func (suite *ScoutConnCollectedTestSuite) TestAddTimestampsAreMonotonic() {
time.Sleep(time.Microsecond)
collected.Add(tls.TypeApplicationData, 100)
for i := 1; i < len(collected.data); i++ {
suite.True(collected.data[i].timestamp.After(collected.data[i-1].timestamp))
data, _ := collected.Snapshot()
for i := 1; i < len(data); i++ {
suite.True(data[i].timestamp.After(data[i-1].timestamp))
}
}
func (suite *ScoutConnCollectedTestSuite) TestConcurrentAddSnapshot() {
collected := NewScoutConnCollected()
var wg sync.WaitGroup
wg.Add(3)
go func() {
defer wg.Done()
for i := 0; i < 1000; i++ {
collected.Add(tls.TypeApplicationData, i)
}
}()
go func() {
defer wg.Done()
for i := 0; i < 100; i++ {
collected.MarkWrite()
}
}()
go func() {
defer wg.Done()
for i := 0; i < 1000; i++ {
// call Snapshot concurrently to exercise the lock under -race
collected.Snapshot() //nolint:errcheck
}
}()
wg.Wait()
data, writeIndex := collected.Snapshot()
suite.Len(data, 1000)
suite.GreaterOrEqual(writeIndex, 0)
}
func TestScoutConnCollected(t *testing.T) {
t.Parallel()
suite.Run(t, &ScoutConnCollectedTestSuite{})
+32 -80
View File
@@ -6,13 +6,12 @@ import (
"crypto/sha256"
"crypto/subtle"
"encoding/binary"
"errors"
"fmt"
"io"
"net"
"slices"
"time"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
const (
@@ -22,12 +21,19 @@ const (
// record_type(1) + version(2) + size(2) + handshake_type(1) + uint24_length(3) + client_version(2)
RandomOffset = 1 + 2 + 2 + 1 + 3 + 2
// https://datatracker.ietf.org/doc/html/rfc8701#name-grease-values
// https://medium.com/asecuritysite-when-bob-met-alice/in-cybersecurity-what-is-grease-9f8850558dea
GreaseMask = 0x0f0f
GreaseValueType = 0x0a0a
sniDNSNamesListType = 0
)
var (
emptyRandom = [RandomLen]byte{}
extTypeSNI = [2]byte{}
ErrCannotFindCipher = errors.New("cannot find a cipher")
)
type ClientHello struct {
@@ -42,11 +48,6 @@ func ReadClientHello(
hostname string,
tolerateTimeSkewness time.Duration,
) (*ClientHello, error) {
if err := conn.SetReadDeadline(time.Now().Add(ClientHelloReadTimeout)); err != nil {
return nil, fmt.Errorf("cannot set read deadline: %w", err)
}
defer conn.SetReadDeadline(resetDeadline) //nolint: errcheck
// This is how FakeTLS is organized:
// 1. We create sha256 HMAC with a given secret
// 2. We dump there a whole TLS frame except of the fact that random
@@ -56,25 +57,17 @@ func ReadClientHello(
// 4. New digest should be all 0 except of last 4 bytes
// 5. Last 4 bytes are little endian uint32 of UNIX timestamp when
// this message was created.
handshakeCopyBuf := &bytes.Buffer{}
reader := io.TeeReader(conn, handshakeCopyBuf)
reader, err := parseTLSHeader(reader)
clientHelloCopy, handshakeReader, err := parseClientHello(conn)
if err != nil {
return nil, fmt.Errorf("cannot parse tls header: %w", err)
return nil, fmt.Errorf("cannot read client hello: %w", err)
}
reader, err = parseHandshakeHeader(reader)
if err != nil {
return nil, fmt.Errorf("cannot parse handshake header: %w", err)
}
hello, err := parseHandshake(reader)
hello, err := parseHandshake(handshakeReader)
if err != nil {
return nil, fmt.Errorf("cannot parse handshake: %w", err)
}
sniHostnames, err := parseSNI(reader)
sniHostnames, err := parseSNI(handshakeReader)
if err != nil {
return nil, fmt.Errorf("cannot parse SNI: %w", err)
}
@@ -85,10 +78,10 @@ func ReadClientHello(
digest := hmac.New(sha256.New, secret)
// we write a copy of the handshake with client random all nullified.
digest.Write(handshakeCopyBuf.Next(RandomOffset))
handshakeCopyBuf.Next(RandomLen)
digest.Write(clientHelloCopy.Next(RandomOffset))
clientHelloCopy.Next(RandomLen)
digest.Write(emptyRandom[:])
digest.Write(handshakeCopyBuf.Bytes())
digest.Write(clientHelloCopy.Bytes())
computed := digest.Sum(nil)
@@ -110,58 +103,6 @@ func ReadClientHello(
return hello, nil
}
func parseTLSHeader(r io.Reader) (io.Reader, error) {
// record_type(1) + version(2) + size(2)
// 16 - type is 0x16 (handshake record)
// 03 01 - protocol version is "3,1" (also known as TLS 1.0)
// 00 f8 - 0xF8 (248) bytes of handshake message follows
header := [1 + 2 + 2]byte{}
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read record header: %w", err)
}
if header[0] != tls.TypeHandshake {
return nil, fmt.Errorf("unexpected record type %#x", header[0])
}
if header[1] != 3 || header[2] != 1 {
return nil, fmt.Errorf("unexpected protocol version %#x %#x", header[1], header[2])
}
length := int64(binary.BigEndian.Uint16(header[3:]))
buf := &bytes.Buffer{}
_, err := io.CopyN(buf, r, length)
return buf, err
}
func parseHandshakeHeader(r io.Reader) (io.Reader, error) {
// type(1) + size(3 / uint24)
// 01 - handshake message type 0x01 (client hello)
// 00 00 f4 - 0xF4 (244) bytes of client hello data follows
header := [1 + 3]byte{}
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read handshake header: %w", err)
}
if header[0] != TypeHandshakeClient {
return nil, fmt.Errorf("incorrect handshake type: %#x", header[0])
}
// unfortunately there is not uint24 in golang, so we just reust header
header[0] = 0
length := int64(binary.BigEndian.Uint32(header[:]))
buf := &bytes.Buffer{}
_, err := io.CopyN(buf, r, length)
return buf, err
}
func parseHandshake(r io.Reader) (*ClientHello, error) {
// A protocol version of "3,3" (meaning TLS 1.2) is given.
header := [2]byte{}
@@ -192,16 +133,27 @@ func parseHandshake(r io.Reader) (*ClientHello, error) {
cipherSuiteLen := int64(binary.BigEndian.Uint16(header[:]))
// we do not care about picking up any cipher. we pick the first one,
// so it is always should be present.
// Pick the first non-GREASE cipher suite from the list.
// Real TLS servers never select GREASE values (RFC 8701, pattern 0x?a?a),
// so echoing them back is a trivial DPI fingerprint.
// cipherSuiteLen is in bytes; each cipher suite is 2 bytes.
for range cipherSuiteLen / 2 {
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read first cipher suite: %w", err)
return nil, fmt.Errorf("cannot read cipher suite: %w", err)
}
hello.CipherSuite = binary.BigEndian.Uint16(header[:])
if hello.CipherSuite != 0 {
// do not forget we have to scan until the end
continue
}
if _, err := io.CopyN(io.Discard, r, cipherSuiteLen-2); err != nil {
return nil, fmt.Errorf("cannot skip remaining cipher suites: %w", err)
if cs := binary.BigEndian.Uint16(header[:]); cs&GreaseMask != GreaseValueType {
hello.CipherSuite = cs
}
}
if hello.CipherSuite == 0 {
return nil, ErrCannotFindCipher
}
if _, err := io.ReadFull(r, header[:1]); err != nil {
@@ -12,7 +12,6 @@ import (
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/tls/fake"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
)
@@ -71,11 +70,6 @@ func (suite *ParseClientHelloSnapshotTestSuite) makeConn(data []byte) *parseClie
readBuf: readBuf,
}
connMock.
On("SetReadDeadline", mock.AnythingOfType("time.Time")).
Twice().
Return(nil)
return connMock
}
+290 -23
View File
@@ -2,9 +2,11 @@ package fake_test
import (
"bytes"
cryptotls "crypto/tls"
"encoding/binary"
"errors"
"encoding/json"
"io"
"os"
"testing"
"time"
@@ -12,7 +14,6 @@ import (
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
"github.com/9seconds/mtg/v2/mtglib/internal/tls/fake"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
)
@@ -51,11 +52,6 @@ func (suite *ParseClientHelloTestSuite) SetupTest() {
suite.connMock = &parseClientHelloConnMock{
readBuf: suite.readBuf,
}
suite.connMock.
On("SetReadDeadline", mock.AnythingOfType("time.Time")).
Twice().
Return(nil)
}
func (suite *ParseClientHelloTestSuite) TearDownTest() {
@@ -67,23 +63,11 @@ type ParseClientHello_TLSHeaderTestSuite struct {
}
func (suite *ParseClientHello_TLSHeaderTestSuite) TestEmpty() {
suite.connMock.ExpectedCalls = []*mock.Call{}
suite.connMock.
On("SetReadDeadline", mock.AnythingOfType("time.Time")).
Once().
Return(errors.New("fail"))
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "fail")
suite.ErrorContains(err, "cannot read client hello")
}
func (suite *ParseClientHello_TLSHeaderTestSuite) TestNothing() {
suite.connMock.ExpectedCalls = []*mock.Call{}
suite.connMock.
On("SetReadDeadline", mock.AnythingOfType("time.Time")).
Twice().
Return(nil)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorIs(err, io.EOF)
}
@@ -232,12 +216,13 @@ func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadCipherSuiteLe
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadFirstCipherSuite() {
body := make([]byte, 2+fake.RandomLen+1+2)
body := make([]byte, 2+fake.RandomLen+1+2+1) // cipherSuiteLen=2 but only 1 byte available
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2)
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read first cipher suite")
suite.ErrorContains(err, "cannot read cipher suite")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipRemainingCipherSuites() {
@@ -247,12 +232,27 @@ func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipRemainingCiph
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot skip remaining cipher suites")
suite.ErrorContains(err, "cannot read cipher suite")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotFindCipher() {
// All cipher suites are GREASE values — must return ErrCannotFindCipher.
body := make([]byte, 2+fake.RandomLen+1+2+4+1)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 4)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2:], 0x0a0a)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2+2:], 0x1a1a)
body[2+fake.RandomLen+1+2+4] = 1
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorIs(err, fake.ErrCannotFindCipher)
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadCompressionMethodsLength() {
body := make([]byte, 2+fake.RandomLen+1+2+2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2:], cryptotls.TLS_AES_128_GCM_SHA256)
suite.writeBody(body)
@@ -263,6 +263,7 @@ func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadCompressionMe
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipCompressionMethods() {
body := make([]byte, 2+fake.RandomLen+1+2+2+1)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2:], cryptotls.TLS_AES_128_GCM_SHA256)
body[2+fake.RandomLen+1+2+2] = 1
suite.writeBody(body)
@@ -298,6 +299,7 @@ func (suite *ParseClientHelloSNITestSuite) writeExtensions(extensions []byte) {
// cipherSuite(2) + compressionLen(1) + compression(1) = 41
body := make([]byte, 41)
binary.BigEndian.PutUint16(body[35:], 2)
binary.BigEndian.PutUint16(body[37:], cryptotls.TLS_AES_128_GCM_SHA256)
body[39] = 1
suite.readBuf.Write(body)
@@ -393,3 +395,268 @@ func TestParseClientHelloSNI(t *testing.T) {
t.Parallel()
suite.Run(t, &ParseClientHelloSNITestSuite{})
}
// fragmentTLSRecord splits a single TLS record into n TLS records by
// dividing the payload into roughly equal parts. Each part gets its own
// TLS record header with the same record type and version.
func fragmentTLSRecord(t testing.TB, full []byte, n int) []byte {
t.Helper()
recordType := full[0]
version := full[1:3]
payload := full[tls.SizeHeader:]
chunkSize := len(payload) / n
result := &bytes.Buffer{}
for i := 0; i < n; i++ {
start := i * chunkSize
end := start + chunkSize
if i == n-1 {
end = len(payload)
}
chunk := payload[start:end]
result.WriteByte(recordType)
result.Write(version)
require.NoError(t, binary.Write(result, binary.BigEndian, uint16(len(chunk))))
result.Write(chunk)
}
return result.Bytes()
}
// splitPayloadAt creates two TLS records from a single record by splitting
// the payload at the given byte position.
func splitPayloadAt(t testing.TB, full []byte, pos int) []byte {
t.Helper()
payload := full[tls.SizeHeader:]
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(t, binary.Write(buf, binary.BigEndian, uint16(pos)))
buf.Write(payload[:pos])
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(t, binary.Write(buf, binary.BigEndian, uint16(len(payload)-pos)))
buf.Write(payload[pos:])
return buf.Bytes()
}
type ParseClientHelloFragmentedTestSuite struct {
suite.Suite
secret mtglib.Secret
snapshot *clientHelloSnapshot
}
func (s *ParseClientHelloFragmentedTestSuite) SetupSuite() {
parsed, err := mtglib.ParseSecret(
"ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d",
)
require.NoError(s.T(), err)
s.secret = parsed
fileData, err := os.ReadFile("testdata/client-hello-ok-19dfe38384b9884b.json")
require.NoError(s.T(), err)
s.snapshot = &clientHelloSnapshot{}
require.NoError(s.T(), json.Unmarshal(fileData, s.snapshot))
}
func (s *ParseClientHelloFragmentedTestSuite) makeConn(data []byte) *parseClientHelloConnMock {
readBuf := &bytes.Buffer{}
readBuf.Write(data)
connMock := &parseClientHelloConnMock{
readBuf: readBuf,
}
return connMock
}
func (s *ParseClientHelloFragmentedTestSuite) TestReassemblySuccess() {
full := s.snapshot.GetFull()
tests := []struct {
name string
data []byte
}{
{"two equal fragments", fragmentTLSRecord(s.T(), full, 2)},
{"three equal fragments", fragmentTLSRecord(s.T(), full, 3)},
{"single byte first fragment", splitPayloadAt(s.T(), full, 1)},
{"three byte first fragment", splitPayloadAt(s.T(), full, 3)},
}
for _, tt := range tests {
s.Run(tt.name, func() {
connMock := s.makeConn(tt.data)
defer connMock.AssertExpectations(s.T())
hello, err := fake.ReadClientHello(
connMock,
s.secret.Key[:],
s.secret.Host,
TolerateTime,
)
s.Require().NoError(err)
s.Equal(s.snapshot.GetRandom(), hello.Random[:])
s.Equal(s.snapshot.GetSessionID(), hello.SessionID)
s.Equal(uint16(s.snapshot.CipherSuite), hello.CipherSuite)
})
}
}
func (s *ParseClientHelloFragmentedTestSuite) TestReassemblyErrors() {
full := s.snapshot.GetFull()
payload := full[tls.SizeHeader:]
tests := []struct {
name string
buildData func() []byte
errMsg string
}{
{
name: "wrong continuation record type",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Wrong type: application data instead of handshake
buf.WriteByte(tls.TypeApplicationData)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(payload)-10)))
buf.Write(payload[10:])
return buf.Bytes()
},
errMsg: "unexpected record type",
},
{
name: "too many continuation records",
buildData: func() []byte {
// Handshake header claiming 256 bytes, but we only send 1 byte per continuation
handshakePayload := []byte{0x01, 0x00, 0x01, 0x00}
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 1})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(handshakePayload))))
buf.Write(handshakePayload)
for range 11 {
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 1})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(1)))
buf.WriteByte(0xAB)
}
return buf.Bytes()
},
errMsg: "too many fragments",
},
{
name: "zero-length continuation record",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Valid header but zero-length payload
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(0)))
return buf.Bytes()
},
errMsg: "cannot read record header",
},
{
name: "wrong continuation record version",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Wrong version: 3.3 instead of 3.1
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 3})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(payload)-10)))
buf.Write(payload[10:])
return buf.Bytes()
},
errMsg: "unexpected protocol version",
},
{
name: "handshake message too large",
buildData: func() []byte {
// Handshake header claiming 0x010000 (65536) bytes — exceeds 0xFFFF limit
handshakePayload := []byte{0x01, 0x01, 0x00, 0x00}
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 1})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(handshakePayload))))
buf.Write(handshakePayload)
return buf.Bytes()
},
errMsg: "cannot read record header",
},
{
name: "truncated continuation record header",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Connection ends mid-header (only 2 bytes)
buf.WriteByte(tls.TypeHandshake)
buf.WriteByte(3)
return buf.Bytes()
},
errMsg: "cannot read record header",
},
{
name: "truncated continuation record payload",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Claims 100 bytes but no payload follows
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(100)))
return buf.Bytes()
},
errMsg: "EOF",
},
}
for _, tt := range tests {
s.Run(tt.name, func() {
connMock := s.makeConn(tt.buildData())
defer connMock.AssertExpectations(s.T())
_, err := fake.ReadClientHello(
connMock,
s.secret.Key[:],
s.secret.Host,
TolerateTime,
)
s.ErrorContains(err, tt.errMsg)
})
}
}
func TestParseClientHelloFragmented(t *testing.T) {
t.Parallel()
suite.Run(t, &ParseClientHelloFragmentedTestSuite{})
}
+1 -10
View File
@@ -2,15 +2,6 @@ package fake
import (
"errors"
"time"
)
const (
ClientHelloReadTimeout = 5 * time.Second
)
var (
resetDeadline time.Time
ErrBadDigest = errors.New("incorrect client random")
)
var ErrBadDigest = errors.New("incorrect client random")
+1 -1
View File
@@ -58,7 +58,7 @@ func (suite *SendServerHelloTestSuite) TestRecordStructure() {
recordType, length, err := tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
suite.Equal(byte(tls.TypeApplicationData), recordType)
suite.Greater(length, int64(2500))
suite.GreaterOrEqual(length, int64(2500))
suite.Empty(suite.buf.Bytes())
}
@@ -0,0 +1,8 @@
{
"time": 1617181365,
"random": "w4TaDfYg/aUKdx1oi68vxMKvHJczRNvtRRppLETzeNE=",
"sessionId": "St2BZ2uHMFn3B2trD1jfdtpjoJOOg6JBeLhFcyCMCq4=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "FgMBAgIBAAH+AwPDhNoN9iD9pQp3HWiLry/Ewq8clzNE2+1FGmksRPN40SBK3YFna4cwWfcHa2sPWN922mOgk46DokF4uEVzIIwKrgA2WloTAxMBEwLALMArwCTAI8AKwAnMqcAwwC/AKMAnwBTAE8yoAJ0AnAA9ADwANQAvwAjAEgAKAQABf/8BAAEAAAAAGwAZAAAWc3RvcmFnZS5nb29nbGVhcGlzLmNvbQAXAAAADQAYABYEAwgEBAEFAwIDCAUIBQUBCAYGAQIBAAUABQEAAAAAM3QAAAASAAAAEAAwAC4CaDIFaDItMTYFaDItMTUFaDItMTQIc3BkeS8zLjEGc3BkeS8zCGh0dHAvMS4xAAsAAgEAADMAJgAkAB0AIAf+6C8fSRJSAC7CyUvdR9kDclNR9KLCsCFHpVZ3bC8iAC0AAgEBACsACQgDBAMDAwIDAQAKAAoACAAdABcAGAAZABUAoQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
}
+158
View File
@@ -0,0 +1,158 @@
package fake
import (
"bytes"
"encoding/binary"
"errors"
"fmt"
"io"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
const (
maxFragmentsCount = 10
)
var ErrTooManyFragments = errors.New("too many fragments")
// https://datatracker.ietf.org/doc/html/rfc5246#section-6.2.1
// client hello can be fragmented in a series of packets:
//
// Bytes on the wire:
//
// 16 03 01 00 F8 01 00 00 F4 03 03 [32 bytes random] [session_id] [ciphers] [SNI...]
// ├─────────────┤├──────────────────────────────────────────────────────────────────┤
//
// TLS record Payload (248 bytes)
// header (5B)
//
// 16 = Handshake
// 03 01 = TLS 1.0 (record layer version)
// 00 F8 = 248 bytes follow
//
// 01 = ClientHello (handshake type)
// 00 00 F4 = 244 bytes of handshake body
// 03 03 = TLS 1.2 (actual protocol version)
// ...rest of ClientHello...
//
// Fragmented record look like:
//
// Record 1:
//
// 16 03 01 00 03 01 00 00
// ├─────────────┤├──────┤
//
// TLS header 3 bytes of payload
//
// 16 = Handshake
// 03 01 = TLS 1.0
// 00 03 = only 3 bytes follow
//
// 01 = ClientHello type
// 00 00 = first 2 bytes of the uint24 length (INCOMPLETE!)
//
// Record 2:
// 16 03 01 00 F5 F4 03 03 [32 bytes random] [session_id] [ciphers] [SNI...]
// ├─────────────┤├────────────────────────────────────────────────────────────┤
//
// TLS header remaining 245 bytes of payload
//
// 16 = Handshake
// 03 01 = TLS 1.0
// 00 F5 = 245 bytes follow
//
// F4 = last byte of uint24 length (now complete: 00 00 F4 = 244)
// 03 03 = TLS 1.2
// ...rest of ClientHello continues...
//
// So it means that there could be a series of handshake packets of different
// lengths. The goal of this function is to concatenate these fragments.
type fragmentedHandshakeReader struct {
r io.Reader
buf bytes.Buffer
readFragments int
}
func (f *fragmentedHandshakeReader) Read(p []byte) (int, error) {
if n, err := f.buf.Read(p); err == nil {
return n, nil
}
f.buf.Reset()
for f.buf.Len() == 0 {
if f.readFragments > maxFragmentsCount {
return 0, ErrTooManyFragments
}
if err := f.parseNextFragment(); err != nil {
return 0, err
}
f.readFragments++
}
return f.buf.Read(p)
}
func (f *fragmentedHandshakeReader) parseNextFragment() error {
// record_type(1) + version(2) + size(2)
// 16 - type is 0x16 (handshake record)
// 03 01 - protocol version is "3,1" (also known as TLS 1.0)
// 00 f8 - 0xF8 (248) bytes of handshake message follows
header := [1 + 2 + 2]byte{}
if _, err := io.ReadFull(f.r, header[:]); err != nil {
return fmt.Errorf("cannot read record header: %w", err)
}
if header[0] != tls.TypeHandshake {
return fmt.Errorf("unexpected record type %#x", header[0])
}
if header[1] != 3 || header[2] != 1 {
return fmt.Errorf("unexpected protocol version %#x %#x", header[1], header[2])
}
length := int64(binary.BigEndian.Uint16(header[3:]))
_, err := io.CopyN(&f.buf, f.r, length)
return err
}
func parseClientHello(r io.Reader) (*bytes.Buffer, *bytes.Buffer, error) {
r = &fragmentedHandshakeReader{r: r}
header := [1 + 3]byte{}
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, nil, fmt.Errorf("cannot read handshake header: %w", err)
}
if header[0] != TypeHandshakeClient {
return nil, nil, fmt.Errorf("incorrect handshake type: %#x", header[0])
}
// unfortunately there is not uint24 in golang, so we just reuse header
header[0] = 0
length := int64(binary.BigEndian.Uint32(header[:]))
clientHelloCopy := &bytes.Buffer{}
clientHelloCopy.Write([]byte{tls.TypeHandshake, 3, 1})
binary.Write( //nolint: errcheck
clientHelloCopy,
binary.BigEndian,
// 1 for handshake type
// 3 for handshake length
uint16(1+3+length),
)
clientHelloCopy.WriteByte(TypeHandshakeClient)
clientHelloCopy.Write(header[1:])
handshakeCopy := &bytes.Buffer{}
writer := io.MultiWriter(clientHelloCopy, handshakeCopy)
_, err := io.CopyN(writer, r, length)
return clientHelloCopy, handshakeCopy, err
}
+20 -4
View File
@@ -28,6 +28,7 @@ type Proxy struct {
allowFallbackOnUnknownDC bool
tolerateTimeSkewness time.Duration
idleTimeout time.Duration
handshakeTimeout time.Duration
domainFrontingPort int
domainFrontingIP string
domainFrontingProxyProtocol bool
@@ -66,6 +67,11 @@ func (p *Proxy) ServeConn(conn essentials.Conn) {
ctx := newStreamContext(p.ctx, p.logger, conn)
defer ctx.Close()
if err := ctx.clientConn.SetDeadline(time.Now().Add(p.handshakeTimeout)); err != nil {
ctx.logger.WarningError("cannot set handshake timeout", err)
return
}
stop := context.AfterFunc(ctx, func() {
ctx.Close()
})
@@ -97,16 +103,23 @@ func (p *Proxy) ServeConn(conn essentials.Conn) {
return
}
if err := ctx.clientConn.SetDeadline(time.Time{}); err != nil {
ctx.logger.WarningError("cannot set deadline", err)
return
}
if err := p.doTelegramCall(ctx); err != nil {
ctx.logger.WarningError("cannot dial to telegram", err)
return
}
tracker := newIdleTracker(p.idleTimeout)
relay.Relay(
ctx,
ctx.logger.Named("relay"),
connIdleTimeout{Conn: ctx.telegramConn, timeout: p.idleTimeout},
connIdleTimeout{Conn: ctx.clientConn, timeout: p.idleTimeout},
connIdleTimeout{Conn: ctx.telegramConn, tracker: tracker},
connIdleTimeout{Conn: ctx.clientConn, tracker: tracker},
)
}
@@ -305,11 +318,13 @@ func (p *Proxy) doDomainFronting(ctx *streamContext, conn *connRewind) {
stream: p.eventStream,
}
tracker := newIdleTracker(p.idleTimeout)
relay.Relay(
ctx,
ctx.logger.Named("domain-fronting"),
connIdleTimeout{Conn: frontConn, timeout: p.idleTimeout},
connIdleTimeout{Conn: conn, timeout: p.idleTimeout},
connIdleTimeout{Conn: frontConn, tracker: tracker},
connIdleTimeout{Conn: conn, tracker: tracker},
)
}
@@ -342,6 +357,7 @@ func NewProxy(opts ProxyOpts) (*Proxy, error) {
domainFrontingIP: opts.DomainFrontingIP,
tolerateTimeSkewness: opts.getTolerateTimeSkewness(),
idleTimeout: opts.getIdleTimeout(),
handshakeTimeout: opts.getHandshakeTimeout(),
allowFallbackOnUnknownDC: opts.AllowFallbackOnUnknownDC,
telegram: tg,
doppelGanger: doppel.NewGanger(
+15 -2
View File
@@ -70,6 +70,12 @@ type ProxyOpts struct {
// This is an optional setting.
IdleTimeout time.Duration
// HandshakeTimeout is a timeout during which all handshake ceremonies must
// be completed, otherwise this process will be aborted
//
// This is an optional setting.
HandshakeTimeout time.Duration
// TolerateTimeSkewness is a time boundary that defines a time range where
// faketls timestamp is acceptable.
//
@@ -160,7 +166,6 @@ type ProxyOpts struct {
// DoppelGangerDRS defines if TLS Dynamic Record Sizing is active.
DoppelGangerDRS bool
}
func (p ProxyOpts) valid() error {
@@ -216,9 +221,17 @@ func (p ProxyOpts) getPreferIP() string {
return p.PreferIP
}
func (p ProxyOpts) getHandshakeTimeout() time.Duration {
if p.HandshakeTimeout == 0 {
return DefaultHandshakeTimeout
}
return p.HandshakeTimeout
}
func (p ProxyOpts) getIdleTimeout() time.Duration {
if p.IdleTimeout == 0 {
return time.Minute
return DefaultIdleTimeout
}
return p.IdleTimeout
+1 -1
View File
@@ -175,7 +175,7 @@ func (suite *ProxyTestSuite) TestHTTPSRequest() {
addr := fmt.Sprintf("https://%s/headers", suite.ProxyAddress())
resp, err := client.Get(addr) //nolint: noctx
suite.NoError(err)
suite.Require().NoError(err)
defer resp.Body.Close() //nolint: errcheck
+14
View File
@@ -36,8 +36,22 @@ const (
// DefaultTCPKeepAlivePeriod defines a time period between 2 consequitive
// probes.
//
// Deprecated: use DefaultKeepAliveIdle and DefaultKeepAliveInterval instead.
DefaultTCPKeepAlivePeriod = 10 * time.Second
// DefaultKeepAliveIdle is the time a connection must be idle before
// the first keepalive probe is sent.
DefaultKeepAliveIdle = 30 * time.Second
// DefaultKeepAliveInterval is the time between consecutive keepalive
// probes.
DefaultKeepAliveInterval = 10 * time.Second
// DefaultKeepAliveCount is the number of unacknowledged probes before
// the connection is considered dead.
DefaultKeepAliveCount = 3
// ProxyDialerOpenThreshold is used for load balancing SOCKS5 dialer only.
//
// This dialer uses circuit breaker with of 3 stages: OPEN, HALF_OPEN and
+7 -2
View File
@@ -20,8 +20,13 @@ func SetServerSocketOptions(conn net.Conn, bufferSize int) error {
}
func setCommonSocketOptions(conn *net.TCPConn) error {
if err := conn.SetKeepAlivePeriod(DefaultTCPKeepAlivePeriod); err != nil {
return fmt.Errorf("cannot set time period of TCP keepalive probes: %w", err)
if err := applyKeepAlive(conn, net.KeepAliveConfig{
Enable: true,
Idle: DefaultKeepAliveIdle,
Interval: DefaultKeepAliveInterval,
Count: DefaultKeepAliveCount,
}); err != nil {
return fmt.Errorf("cannot configure TCP keepalive: %w", err)
}
if err := conn.SetLinger(tcpLingerTimeout); err != nil {
+11
View File
@@ -0,0 +1,11 @@
//go:build !openbsd
package network
import "net"
// applyKeepAlive enables TCP keepalive on conn and applies the per-socket
// idle/interval/count tuning from cfg.
func applyKeepAlive(conn *net.TCPConn, cfg net.KeepAliveConfig) error {
return conn.SetKeepAliveConfig(cfg) //nolint: wrapcheck
}
+20
View File
@@ -0,0 +1,20 @@
package network
import "net"
// applyKeepAlive enables (or disables) TCP keepalive on conn.
//
// OpenBSD has no user-settable per-socket TCP keepalive options: TCP_KEEPIDLE,
// TCP_KEEPINTVL and TCP_KEEPCNT do not exist on OpenBSD, and Go's
// (*TCPConn).SetKeepAliveConfig therefore returns ENOPROTOOPT ("protocol not
// available") for any non-negative Idle/Interval/Count value (see
// src/net/tcpsockopt_openbsd.go in the Go source tree). Calling
// SetKeepAliveConfig with mtg's defaults (zero values) breaks every accepted
// listener connection and every outbound dial on OpenBSD.
//
// On OpenBSD we only flip SO_KEEPALIVE on or off; the keepalive timing is
// controlled system-wide via the sysctl knobs net.inet.tcp.keepidle and
// net.inet.tcp.keepintvl.
func applyKeepAlive(conn *net.TCPConn, cfg net.KeepAliveConfig) error {
return conn.SetKeepAlive(cfg.Enable) //nolint: wrapcheck
}
+93
View File
@@ -0,0 +1,93 @@
//go:build linux || darwin
// +build linux darwin
package network_test
import (
"net"
"runtime"
"syscall"
"testing"
"time"
"github.com/9seconds/mtg/v2/network"
"github.com/stretchr/testify/require"
"golang.org/x/sys/unix"
)
func tcpKeepIdleOption() int {
if runtime.GOOS == "darwin" {
return 0x10 // TCP_KEEPALIVE on macOS
}
return 0x4 // TCP_KEEPIDLE on Linux
}
func TestSetClientSocketOptionsKeepAlive(t *testing.T) {
t.Parallel()
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() {
err := listener.Close()
require.NoError(t, err)
}()
type dialResult struct {
conn net.Conn
err error
}
dialDone := make(chan dialResult, 1)
go func() {
c, err := net.Dial("tcp", listener.Addr().String())
dialDone <- dialResult{conn: c, err: err}
}()
tcpListener, ok := listener.(*net.TCPListener)
require.True(t, ok, "listener must be a *net.TCPListener")
require.NoError(t, tcpListener.SetDeadline(time.Now().Add(5*time.Second)))
accepted, err := listener.Accept()
require.NoError(t, err)
defer func() {
err := accepted.Close()
require.NoError(t, err)
}()
dr := <-dialDone
require.NoError(t, dr.err)
defer func() {
err := dr.conn.Close()
require.NoError(t, err)
}()
err = network.SetClientSocketOptions(accepted, 0)
require.NoError(t, err)
tcpConn := accepted.(*net.TCPConn)
rawConn, err := tcpConn.SyscallConn()
require.NoError(t, err)
err = rawConn.Control(func(fd uintptr) {
val, err := unix.GetsockoptInt(int(fd), syscall.SOL_SOCKET, syscall.SO_KEEPALIVE)
require.NoError(t, err)
require.NotEqual(t, 0, val, "SO_KEEPALIVE should be enabled")
idle, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, tcpKeepIdleOption())
require.NoError(t, err)
require.Equal(t, int(network.DefaultKeepAliveIdle.Seconds()), idle)
interval, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, unix.TCP_KEEPINTVL)
require.NoError(t, err)
require.Equal(t, int(network.DefaultKeepAliveInterval.Seconds()), interval)
count, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, unix.TCP_KEEPCNT)
require.NoError(t, err)
require.Equal(t, network.DefaultKeepAliveCount, count)
})
require.NoError(t, err)
}
+1 -1
View File
@@ -25,7 +25,7 @@ func (suite *BaseHTTPTestSuite) SetupSuite() {
}
func (suite *BaseHTTPTestSuite) SetupTest() {
suite.client = network.New(nil, "mtg/1", 0, 0, 0).MakeHTTPClient(nil)
suite.client = network.New(nil, "mtg/1", 0, 0, 0, network.DefaultKeepAliveConfig).MakeHTTPClient(nil)
}
func (suite *BaseHTTPTestSuite) TestGet() {
+1 -1
View File
@@ -19,7 +19,7 @@ type BaseNetworkTestSuite struct {
func (suite *BaseNetworkTestSuite) SetupSuite() {
suite.EchoServerTestSuite.SetupSuite()
suite.net = network.New(nil, "agent", 0, 0, 0)
suite.net = network.New(nil, "agent", 0, 0, 0, network.DefaultKeepAliveConfig)
}
func (suite *BaseNetworkTestSuite) TestDialUnknownNetwork() {
+43 -1
View File
@@ -11,6 +11,7 @@ package network
import (
"errors"
"net"
"time"
)
@@ -26,14 +27,55 @@ const (
// DefaultTCPKeepAlivePeriod defines a time period between 2 consecuitive
// probes.
//
// Deprecated: use DefaultKeepAliveConfig
DefaultTCPKeepAlivePeriod = 10 * time.Second
// DefaultKeepAliveIdle is the time a connection must be idle before
// the first keepalive probe is sent.
//
// Deprecated: use DefaultKeepAliveConfig
DefaultKeepAliveIdle = 30 * time.Second
// DefaultKeepAliveInterval is the time between consecutive keepalive
// probes.
//
// Deprecated: use DefaultKeepAliveConfig
DefaultKeepAliveInterval = 10 * time.Second
// DefaultKeepAliveCount is the number of unacknowledged probes before
// the connection is considered dead.
//
// Deprecated: use DefaultKeepAliveConfig
DefaultKeepAliveCount = 3
// User Agent to use in HTTP client.
UserAgent = "curl/8.5.0"
// tcpLingerTimeout defines a number of seconds to wait for sending
// unacknowledged data.
tcpLingerTimeout = 1
// tcpNotSentLowat limits the amount of unsent data queued in the
// kernel write buffer per socket. When the unsent data drops below
// this threshold, the socket becomes writable again. This reduces
// per-connection memory usage and bufferbloat by applying
// back-pressure to the relay loop instead of piling up data in
// kernel buffers.
tcpNotSentLowat = 128 * 1024
)
var ErrCannotDial = errors.New("cannot dial to any address")
var (
ErrCannotDial = errors.New("cannot dial to any address")
// DefaultKeepAliveConfig defines a default configuration for
// keep alive settings. As per official documentation, if keep alive
// is enabled, then:
//
// Idle = 15 * time.Second
// Interval = 15 * time.Second
// Count = 9
DefaultKeepAliveConfig = net.KeepAliveConfig{
Enable: true,
}
)
+4 -1
View File
@@ -14,6 +14,7 @@ import (
type network struct {
net.Dialer
keepAliveConfig net.KeepAliveConfig
httpTimeout time.Duration
idleTimeout time.Duration
userAgent string
@@ -37,7 +38,7 @@ func (n *network) DialContext(ctx context.Context, network, address string) (ess
tcpConn := conn.(*net.TCPConn)
return tcpConn, setCommonSocketOptions(tcpConn)
return tcpConn, setCommonSocketOptions(tcpConn, n.keepAliveConfig)
}
func (n *network) MakeHTTPClient(
@@ -71,6 +72,7 @@ func New(
tcpTimeout,
httpTimeout,
idleTimeout time.Duration,
keepAliveConfig net.KeepAliveConfig,
) mtglib.Network {
if dnsResolver == nil {
dnsResolver = net.DefaultResolver
@@ -89,5 +91,6 @@ func New(
userAgent: userAgent,
idleTimeout: idleTimeout,
httpTimeout: httpTimeout,
keepAliveConfig: keepAliveConfig,
}
}
+7 -3
View File
@@ -5,9 +5,9 @@ import (
"net"
)
func setCommonSocketOptions(conn *net.TCPConn) error {
if err := conn.SetKeepAlivePeriod(DefaultTCPKeepAlivePeriod); err != nil {
return fmt.Errorf("cannot set time period of TCP keepalive probes: %w", err)
func setCommonSocketOptions(conn *net.TCPConn, keepAliveConfig net.KeepAliveConfig) error {
if err := applyKeepAlive(conn, keepAliveConfig); err != nil {
return fmt.Errorf("cannot configure TCP keepalive: %w", err)
}
if err := conn.SetLinger(tcpLingerTimeout); err != nil {
@@ -23,5 +23,9 @@ func setCommonSocketOptions(conn *net.TCPConn) error {
return fmt.Errorf("cannot setup SO_REUSEADDR/PORT: %w", err)
}
setCongestionControl(rawConn)
setTCPUserTimeout(rawConn, keepAliveConfig)
setNotSentLowat(rawConn)
return nil
}
+20
View File
@@ -0,0 +1,20 @@
//go:build linux
package network
import (
"syscall"
"golang.org/x/sys/unix"
)
// setCongestionControl sets BBR as the TCP congestion control algorithm.
// BBR provides better throughput over lossy and high-latency links compared
// to the default cubic, which is especially beneficial for mobile and
// home internet clients. This is best-effort: silently ignored if the
// kernel does not have tcp_bbr available.
func setCongestionControl(conn syscall.RawConn) {
conn.Control(func(fd uintptr) { //nolint: errcheck
unix.SetsockoptString(int(fd), unix.IPPROTO_TCP, unix.TCP_CONGESTION, "bbr") //nolint: errcheck
})
}
+7
View File
@@ -0,0 +1,7 @@
//go:build !linux
package network
import "syscall"
func setCongestionControl(conn syscall.RawConn) {}
+11
View File
@@ -0,0 +1,11 @@
//go:build !openbsd
package network
import "net"
// applyKeepAlive enables TCP keepalive on conn and applies the per-socket
// idle/interval/count tuning from cfg.
func applyKeepAlive(conn *net.TCPConn, cfg net.KeepAliveConfig) error {
return conn.SetKeepAliveConfig(cfg) //nolint: wrapcheck
}
+20
View File
@@ -0,0 +1,20 @@
package network
import "net"
// applyKeepAlive enables (or disables) TCP keepalive on conn.
//
// OpenBSD has no user-settable per-socket TCP keepalive options: TCP_KEEPIDLE,
// TCP_KEEPINTVL and TCP_KEEPCNT do not exist on OpenBSD, and Go's
// (*TCPConn).SetKeepAliveConfig therefore returns ENOPROTOOPT ("protocol not
// available") for any non-negative Idle/Interval/Count value (see
// src/net/tcpsockopt_openbsd.go in the Go source tree). Calling
// SetKeepAliveConfig with mtg's defaults (zero values) breaks every accepted
// listener connection and every outbound dial on OpenBSD.
//
// On OpenBSD we only flip SO_KEEPALIVE on or off; the keepalive timing is
// controlled system-wide via the sysctl knobs net.inet.tcp.keepidle and
// net.inet.tcp.keepintvl.
func applyKeepAlive(conn *net.TCPConn, cfg net.KeepAliveConfig) error {
return conn.SetKeepAlive(cfg.Enable) //nolint: wrapcheck
}
+20
View File
@@ -0,0 +1,20 @@
//go:build linux || darwin
package network
import (
"syscall"
"golang.org/x/sys/unix"
)
// setNotSentLowat sets TCP_NOTSENT_LOWAT which limits the amount of
// unsent data queued in the kernel write buffer. Once unsent data drops
// below this threshold the socket becomes writable again, applying
// back-pressure to the relay loop instead of piling up data in kernel
// buffers. This reduces per-connection memory and bufferbloat.
func setNotSentLowat(conn syscall.RawConn) {
conn.Control(func(fd uintptr) { //nolint: errcheck
unix.SetsockoptInt(int(fd), unix.IPPROTO_TCP, unix.TCP_NOTSENT_LOWAT, tcpNotSentLowat) //nolint: errcheck
})
}
+7
View File
@@ -0,0 +1,7 @@
//go:build !linux && !darwin
package network
import "syscall"
func setNotSentLowat(conn syscall.RawConn) {}
@@ -1,5 +1,4 @@
//go:build !windows
// +build !windows
package network
@@ -1,5 +1,4 @@
//go:build windows
// +build windows
package network
+92
View File
@@ -0,0 +1,92 @@
//go:build linux || darwin
// +build linux darwin
package network
import (
"net"
"runtime"
"syscall"
"testing"
"time"
"github.com/stretchr/testify/require"
"golang.org/x/sys/unix"
)
func tcpKeepIdleOption() int {
if runtime.GOOS == "darwin" {
return 0x10 // TCP_KEEPALIVE on macOS
}
return 0x4 // TCP_KEEPIDLE on Linux
}
func TestSetCommonSocketOptionsKeepAlive(t *testing.T) {
t.Parallel()
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() {
err := listener.Close()
require.NoError(t, err)
}()
type dialResult struct {
conn net.Conn
err error
}
dialDone := make(chan dialResult, 1)
go func() {
c, err := net.Dial("tcp", listener.Addr().String())
dialDone <- dialResult{conn: c, err: err}
}()
tcpListener, ok := listener.(*net.TCPListener)
require.True(t, ok, "listener must be a *net.TCPListener")
require.NoError(t, tcpListener.SetDeadline(time.Now().Add(5*time.Second)))
accepted, err := listener.Accept()
require.NoError(t, err)
defer func() {
err := accepted.Close()
require.NoError(t, err)
}()
dr := <-dialDone
require.NoError(t, dr.err)
defer func() {
err := dr.conn.Close()
require.NoError(t, err)
}()
tcpConn := accepted.(*net.TCPConn)
err = setCommonSocketOptions(tcpConn, DefaultKeepAliveConfig)
require.NoError(t, err)
rawConn, err := tcpConn.SyscallConn()
require.NoError(t, err)
err = rawConn.Control(func(fd uintptr) {
val, err := unix.GetsockoptInt(int(fd), syscall.SOL_SOCKET, syscall.SO_KEEPALIVE)
require.NoError(t, err)
require.NotEqual(t, 0, val, "SO_KEEPALIVE should be enabled")
idle, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, tcpKeepIdleOption())
require.NoError(t, err)
require.Equal(t, 15, idle, "keepalive idle should match DefaultKeepAliveIdle")
interval, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, unix.TCP_KEEPINTVL)
require.NoError(t, err)
require.Equal(t, 15, interval, "keepalive interval should match DefaultKeepAliveInterval")
count, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, unix.TCP_KEEPCNT)
require.NoError(t, err)
require.Equal(t, 9, count, "keepalive count should match DefaultKeepAliveCount")
})
require.NoError(t, err)
}
+48
View File
@@ -0,0 +1,48 @@
//go:build linux
package network
import (
"net"
"syscall"
"time"
"golang.org/x/sys/unix"
)
// Go runtime defaults for KeepAliveConfig when fields are zero.
const (
goDefaultKeepAliveIdle = 15 * time.Second
goDefaultKeepAliveInterval = 15 * time.Second
goDefaultKeepAliveCount = 9
)
// setTCPUserTimeout sets TCP_USER_TIMEOUT on a socket. If transmitted
// data remains unacknowledged for this long, the kernel closes the
// connection. As recommended by Cloudflare
// (https://blog.cloudflare.com/when-tcp-sockets-refuse-to-die/),
// the value is computed as: keepidle + keepintvl * keepcnt. This
// ensures TCP_USER_TIMEOUT and keepalives agree on when to give up.
// Best-effort: silently ignored if unsupported.
func setTCPUserTimeout(conn syscall.RawConn, cfg net.KeepAliveConfig) {
idle := cfg.Idle
if idle == 0 {
idle = goDefaultKeepAliveIdle
}
interval := cfg.Interval
if interval == 0 {
interval = goDefaultKeepAliveInterval
}
count := cfg.Count
if count == 0 {
count = goDefaultKeepAliveCount
}
timeout := idle + interval*time.Duration(count)
conn.Control(func(fd uintptr) { //nolint: errcheck
unix.SetsockoptInt(int(fd), unix.IPPROTO_TCP, unix.TCP_USER_TIMEOUT, int(timeout.Milliseconds())) //nolint: errcheck
})
}
+10
View File
@@ -0,0 +1,10 @@
//go:build !linux
package network
import (
"net"
"syscall"
)
func setTCPUserTimeout(conn syscall.RawConn, cfg net.KeepAliveConfig) {}
+1 -1
View File
@@ -66,7 +66,7 @@ func (suite *SocksProxyTestSuite) SetupSuite() {
require.NoError(suite.T(), err)
suite.authURL = parsed
suite.baseNetwork = network.New(nil, "mtg", 0, 0, 0)
suite.baseNetwork = network.New(nil, "mtg", 0, 0, 0, network.DefaultKeepAliveConfig)
}
func (suite *SocksProxyTestSuite) TestIncorrectSchema() {