REPOSITORY / ScuroNeko/mtg

Compare commits

DIFF REPOSITORY

Compare commits

..
27 Commits
Author SHA1 Message Date
9seconds 8b3c622ea6 Merge remote-tracking branch 'origin/stable' into v2 2026-03-29 23:33:24 +02:00
9seconds 9d43c2d759 Merge remote-tracking branch 'origin/master' into stable 2026-03-29 23:33:06 +02:00
9seconds 006fba1046 Merge remote-tracking branch 'origin/stable' into v2 2026-03-24 09:59:03 +01:00
9seconds 7b333ed833 Merge remote-tracking branch 'origin/master' into stable 2026-03-24 09:58:48 +01:00
9seconds de89de2ad6 Merge remote-tracking branch 'origin/master' into stable 2026-03-24 09:57:43 +01:00
9seconds 7fec30908a Merge remote-tracking branch 'origin/stable' into v2 2026-03-20 11:29:39 +01:00
9seconds 2eb0828f72 Merge remote-tracking branch 'origin/master' into stable 2026-03-20 11:29:24 +01:00
9seconds fc72de9e39 Merge remote-tracking branch 'origin/stable' into v2 2026-03-19 18:52:35 +01:00
9seconds cb627f2a66 Merge remote-tracking branch 'origin/master' into stable 2026-03-19 18:52:11 +01:00
9seconds dc81f7981c Merge remote-tracking branch 'origin/stable' into v2 2026-03-16 23:56:10 +01:00
9seconds 9d5fd989e5 Merge remote-tracking branch 'origin/master' into stable 2026-03-16 23:55:56 +01:00
9seconds d0502e7083 Merge remote-tracking branch 'origin/stable' into v2 2026-03-15 22:04:35 +01:00
9seconds 30aa9d3a44 Merge remote-tracking branch 'origin/master' into stable 2026-03-15 22:04:09 +01:00
9seconds fefc479f94 Merge remote-tracking branch 'origin/stable' into v2 2026-02-28 14:52:05 +01:00
9seconds 9f12620dba Merge remote-tracking branch 'origin/master' into stable 2026-02-28 14:51:26 +01:00
9seconds f5244c2bfd Merge remote-tracking branch 'origin/stable' into v2 2026-02-28 14:47:43 +01:00
9seconds 9946cdfeb5 Merge remote-tracking branch 'origin/master' into stable 2026-02-28 14:47:25 +01:00
9seconds 7d748077e9 Merge remote-tracking branch 'origin/stable' into v2 2026-02-26 13:53:32 +01:00
9seconds f61a63a59b Merge remote-tracking branch 'origin/master' into stable 2026-02-26 13:53:17 +01:00
9seconds 136eea551f Merge remote-tracking branch 'origin/stable' into v2 2026-02-24 18:46:18 +01:00
9seconds e6fa5906c9 Merge remote-tracking branch 'origin/master' into stable 2026-02-24 18:45:46 +01:00
9seconds b854b16e1a Merge branch 'stable' into v2 2026-02-19 14:37:52 +01:00
9seconds f4b296d1e0 Merge branch 'master' into stable 2026-02-19 14:37:36 +01:00
9seconds e7fdb02a29 Merge branch 'v2' into stable 2026-02-16 19:23:45 +01:00
9seconds 7a6ba6d8c6 Merge branch 'master' into v2 2026-02-16 19:23:28 +01:00
9seconds 25bca76da5 Merge remote-tracking branch 'origin/stable' into v2 2026-02-11 14:26:31 +01:00
9seconds 76109f6204 Merge remote-tracking branch 'origin/master' into stable 2026-02-11 14:25:54 +01:00
61 changed files with 214 additions and 1787 deletions
+1 -1
View File
@@ -122,7 +122,7 @@ jobs:
artifacts: artifacts:
name: Build release artifacts name: Build release artifacts
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 20 timeout-minutes: 10
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v6 uses: actions/checkout@v6
-2
View File
@@ -8,5 +8,3 @@
mtg mtg
coverage.txt coverage.txt
dist/ dist/
.idea/
.vscode/
+1 -1
View File
@@ -33,7 +33,7 @@ RUN go mod download
COPY . /app COPY . /app
RUN set -x \ RUN set -x \
&& version="$(git describe --exact-match HEAD 2>/dev/null || git describe --tags --always 2>/dev/null || echo dev)" \ && version="$(git describe --exact-match HEAD 2>/dev/null || git describe --tags --always)" \
&& go build \ && go build \
-trimpath \ -trimpath \
-mod=readonly \ -mod=readonly \
-46
View File
@@ -29,8 +29,6 @@ are the most notable:
* [Official](https://github.com/TelegramMessenger/MTProxy) * [Official](https://github.com/TelegramMessenger/MTProxy)
* [Python](https://github.com/alexbers/mtprotoproxy) * [Python](https://github.com/alexbers/mtprotoproxy)
* [Erlang](https://github.com/seriyps/mtproto_proxy) * [Erlang](https://github.com/seriyps/mtproto_proxy)
* [Teleproxy (C)](https://github.com/teleproxy/teleproxy)
* [mtproto.zig (Zig)](https://github.com/sleep3r/mtproto.zig)
* [Telemt (Rust)](https://github.com/telemt/telemt) * [Telemt (Rust)](https://github.com/telemt/telemt)
You can use any of these. They work great and all implementations have You can use any of these. They work great and all implementations have
@@ -514,50 +512,6 @@ This is not very necessary. Keep in mind these rules:
you can enable `drs` setting. you can enable `drs` setting.
9. **If you are not sure, touch nothing!** 9. **If you are not sure, touch nothing!**
## Troubleshooting
### `ip was blacklisted` for clients on the same LAN
If you run mtg at home and a client on the same LAN (for example, your
phone on the home Wi-Fi) cannot connect, check the proxy logs for a
message like:
```json
{"level":"info","ip":"10.0.1.1","logger":"proxy","message":"ip was blacklisted"}
```
The reason is that the default blocklist (`firehol_level1.netset`)
includes bogon networks, which covers all RFC1918 ranges
(`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`). Any client
connecting from such an address is rejected by the blocklist —
the TCP connection is closed immediately with no response, so
from the client's point of view nothing loads at all.
There are three ways to resolve it:
1. Disable the blocklist entirely in `config.toml`:
```toml
[defense.blocklist]
enabled = false
```
Simplest option if the proxy is used only by you and people you trust.
2. Keep the blocklist but swap `firehol_level1` for a narrower list that
does not include bogons, for example `firehol_abusers_1d`:
```toml
[defense.blocklist]
enabled = true
urls = ["https://iplists.firehol.org/files/firehol_abusers_1d.netset"]
```
3. Connect to the proxy through a public IP or domain name with hairpin
NAT (`MASQUERADE`) on your router. mtg will then see the client with
its public address and the blocklist will not match. This is more
work to set up but preserves full blocklist protection.
## Metrics ## Metrics
Out of the box, mtg works with Out of the box, mtg works with
+1 -1
View File
@@ -12,7 +12,7 @@ type StableBloomFilterTestSuite struct {
} }
func (suite *StableBloomFilterTestSuite) TestOp() { func (suite *StableBloomFilterTestSuite) TestOp() {
filter := antireplay.NewStableBloomFilter(100000, 0.001) filter := antireplay.NewStableBloomFilter(500, 0.001)
suite.False(filter.SeenBefore([]byte{1, 2, 3})) suite.False(filter.SeenBefore([]byte{1, 2, 3}))
suite.False(filter.SeenBefore([]byte{4, 5, 6})) suite.False(filter.SeenBefore([]byte{4, 5, 6}))
-30
View File
@@ -1,30 +0,0 @@
{
# Caddy sits behind HAProxy which passes raw TLS through on :8443.
# ACME HTTP-01 challenges arrive on :80 via HAProxy's acl passthrough.
http_port 80
https_port 8443
# HAProxy forwards connections to :8443 with a PROXY protocol v2
# header (see haproxy.cfg `send-proxy-v2`). The proxy_protocol
# listener wrapper strips the header and exposes the real client IP
# to Caddy's access log. The `tls` wrapper must follow so that TLS
# is terminated on the unwrapped connection.
#
# `allow` lists the networks permitted to send PROXY headers. These
# ranges cover docker compose's default bridge networks; tighten
# them if you pin a specific subnet in docker-compose.yml.
servers :8443 {
listener_wrappers {
proxy_protocol {
timeout 5s
allow 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
}
tls
}
}
}
{$DOMAIN} {
root * /srv
file_server
}
-89
View File
@@ -1,89 +0,0 @@
# SNI-routing deployment for mtg
A turnkey `docker compose` setup that puts an SNI-aware TCP router
(HAProxy) in front of mtg **and** a real web server (Caddy with
automatic HTTPS).
## Why
Modern DPI systems actively probe suspected proxies. If the server
closes the connection or returns something unexpected, the IP gets
flagged. With this setup:
- **Telegram clients** connect to port 443, HAProxy sees the configured
SNI and routes them to mtg (FakeTLS).
- **Everything else** (browsers, DPI probes, scanners) is routed to
Caddy, which responds with a real Let's Encrypt certificate and serves
genuine web content.
Because your domain's DNS points to this server, the SNI/IP match is
natural and passive DPI has nothing to flag.
## Quick start
```bash
# 1. Point your domain's DNS A/AAAA record to this server's IP.
# 2. Generate an mtg secret:
docker run --rm nineseconds/mtg:2 generate-secret --hex YOUR_DOMAIN
# 3. Edit the config files:
# - mtg-config.toml → paste the secret
# - haproxy.cfg → replace "example.com" in the SNI ACL
# - .env or export → DOMAIN=your.domain
# 4. (Optional) put your site content into www/
# 5. Start:
docker compose up -d
# 6. Verify:
# - Open https://YOUR_DOMAIN in a browser → you should see the web page
# - Configure Telegram with the proxy link from:
docker compose exec mtg mtg access /config/config.toml
```
## Real client IPs (PROXY protocol)
HAProxy forwards TCP connections to mtg and Caddy with a PROXY protocol
v2 header so both backends see the real client IP instead of HAProxy's
container address. The three pieces must stay in sync:
- `haproxy.cfg``send-proxy-v2` on the `mtg` and `web` backend `server` lines
- `mtg-config.toml``proxy-protocol-listener = true`
- `Caddyfile``listener_wrappers { proxy_protocol { ... } tls }` on `:8443`
If you disable one, disable all three, otherwise the backend will fail
to parse the connection.
## ACME (Let's Encrypt) notes
HAProxy passes `/.well-known/acme-challenge/` requests on `:80` to
Caddy so that HTTP-01 validation works out of the box. Make sure your
domain's DNS A/AAAA record points to this server before starting.
## Architecture
```
┌──────────────────┐
:443 ──────>│ HAProxy │
│ (TCP, SNI peek) │
└──┬───────────┬───┘
SNI match │ │ default
v v
┌─────────┐ ┌─────────┐
│ mtg │ │ Caddy │
│ :3128 │ │ :8443 │
│ FakeTLS │ │ real TLS│
└─────────┘ └─────────┘
```
## Files
| File | Purpose |
|---|---|
| `docker-compose.yml` | Service definitions |
| `haproxy.cfg` | SNI routing rules — **edit the domain** |
| `mtg-config.toml` | mtg proxy config — **paste your secret** |
| `Caddyfile` | Web server config (auto-HTTPS) |
| `www/` | Static site content served by Caddy |
-54
View File
@@ -1,54 +0,0 @@
# SNI-routing deployment: HAProxy (443) -> mtg + real web backend
#
# This setup puts an SNI-aware TCP router in front of mtg so that:
# - Telegram clients (FakeTLS with the correct SNI) are routed to mtg
# - All other TLS traffic (including DPI probes) reaches the real web
# server, which responds with a genuine certificate
#
# The result: active probes see a real website; passive DPI sees matching
# SNI/IP because the domain resolves to this server's IP.
#
# Quick start:
# 1. Set YOUR_DOMAIN below (and in mtg-config.toml)
# 2. docker compose up -d
# 3. mtg generate-secret YOUR_DOMAIN -> put it in mtg-config.toml
# 4. docker compose restart mtg
#
# See BEST_PRACTICES.md and the project wiki for background.
services:
haproxy:
image: haproxy:lts-alpine
ports:
- "443:443"
- "80:80"
volumes:
- ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
depends_on:
- mtg
- web
restart: unless-stopped
mtg:
image: nineseconds/mtg:2
volumes:
- ./mtg-config.toml:/config/config.toml:ro
expose:
- "3128"
restart: unless-stopped
web:
image: caddy:alpine
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy_data:/data
- ./www:/srv:ro
expose:
- "80"
- "8443"
environment:
DOMAIN: ${DOMAIN:-example.com}
restart: unless-stopped
volumes:
caddy_data:
-62
View File
@@ -1,62 +0,0 @@
# HAProxy SNI router — Layer 4 (TCP mode)
#
# Inspects the SNI in the TLS ClientHello and routes traffic:
# - SNI matching the mtg secret domain -> mtg (FakeTLS / MTProto)
# - Everything else -> real web backend (Caddy)
#
# Because routing happens before TLS termination, each backend sees the
# raw ClientHello and handles TLS itself. The real web backend therefore
# presents a genuine certificate to any probe or browser.
global
log stdout format raw local0 info
maxconn 4096
defaults
log global
mode tcp
option tcplog
timeout connect 5s
timeout client 60s
timeout server 60s
# --- HTTP :80 — ACME challenges + redirect -----------------------------------
frontend http
bind *:80
mode http
# Let Caddy answer ACME HTTP-01 challenges for Let's Encrypt.
acl is_acme path_beg /.well-known/acme-challenge/
use_backend web_acme if is_acme
http-request redirect scheme https code 301
# --- TLS :443 — SNI-based routing -------------------------------------------
frontend tls
bind *:443
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
# Route Telegram clients to mtg.
# Replace "example.com" with the domain from your mtg secret.
use_backend mtg if { req_ssl_sni -i example.com }
default_backend web
backend mtg
# send-proxy-v2 prepends a PROXY protocol v2 header so mtg sees the
# real client IP instead of HAProxy's. mtg must have
# `proxy-protocol-listener = true` in its config.
server mtg mtg:3128 send-proxy-v2
backend web
# send-proxy-v2 prepends a PROXY protocol v2 header so Caddy logs the
# real client IP instead of HAProxy's. Caddy must enable the
# proxy_protocol listener wrapper on :8443 (see Caddyfile).
server web web:8443 send-proxy-v2
backend web_acme
mode http
server web web:80
-17
View File
@@ -1,17 +0,0 @@
# Minimal mtg configuration for the SNI-router setup.
#
# 1. Generate a secret: mtg generate-secret --hex example.com
# 2. Paste it below.
# 3. Replace example.com with your actual domain everywhere.
secret = "PASTE_YOUR_SECRET_HERE"
bind-to = "0.0.0.0:3128"
# HAProxy in front sends PROXY protocol v2 headers so mtg can see the
# real client IP. Keep this in sync with haproxy.cfg (`send-proxy-v2`).
proxy-protocol-listener = true
[defense.anti-replay]
enabled = true
max-size = "1mib"
error-rate = 0.001
-5
View File
@@ -1,5 +0,0 @@
<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>Welcome</title></head>
<body><h1>It works!</h1><p>Replace this with your own content.</p></body>
</html>
BIN
View File
Binary file not shown.
+1 -13
View File
@@ -25,18 +25,6 @@ var TelegramCoreAddresses = map[int][]string{
}, },
203: { 203: {
"91.105.192.100:443", "91.105.192.100:443",
"[2a0a:f280:0203:a:5000::100]:443", "[2a0a:f280:0203:000a:5000:0000:0000:0100]:443",
},
10001: {
"149.154.175.10:443",
"[2001:b28:f23d:f001::e]:443",
},
10002: {
"149.154.167.40:443",
"[2001:67c:4e8:f002::e]:443",
},
10003: {
"149.154.175.117:443",
"[2001:b28:f23d:f003::e]:443",
}, },
} }
+5 -32
View File
@@ -204,22 +204,14 @@ proxies = [
# define a global timeout on establishing of network connections. idle # define a global timeout on establishing of network connections. idle
# means a timeout on pumping data between sockset when nothing is # means a timeout on pumping data between sockset when nothing is
# happening. # happening.
#
# please be noticed that handshakes have no timeouts intentionally. You can
# find a reasoning here:
# https://www.ndss-symposium.org/wp-content/uploads/2020/02/23087-paper.pdf
[network.timeout] [network.timeout]
tcp = "5s" tcp = "5s"
http = "10s" http = "10s"
idle = "5m" idle = "1m"
handshake = "10s"
# this defines a configuration for TCP keep alives. Default values are taken
# from Golang default behavior.
[network.keep-alive]
disabled = false
# idle means a time period after which we start sending TCP Keep Alive probes
idle = "15s"
# interval is a period between 2 consecutive probes
interval = "15s"
# if we miss that many probes, a connection will be considered as a dead one.
count = 9
# mtg has to mimic real websites. It does not mean domain fronting, it also # mtg has to mimic real websites. It does not mean domain fronting, it also
# means that traffic characteristics should be similar to real world traffic. # means that traffic characteristics should be similar to real world traffic.
@@ -316,17 +308,6 @@ download-concurrency = 2
# A list of URLs in FireHOL format (https://iplists.firehol.org/) # A list of URLs in FireHOL format (https://iplists.firehol.org/)
# You can provider links here (starts with https:// or http://) or # You can provider links here (starts with https:// or http://) or
# path to a local file, but in this case it should be absolute. # path to a local file, but in this case it should be absolute.
#
# NOTE: the default list below (firehol_level1.netset) includes bogon
# networks, and therefore RFC1918 ranges as well (10.0.0.0/8,
# 172.16.0.0/12, 192.168.0.0/16). If you run mtg on a home/LAN network
# and connect from a client on the same LAN, that client will be
# rejected with "ip was blacklisted" and the connection dropped (TCP
# close, no response). If you see this, you can either disable this section
# (enabled = false), replace firehol_level1 with a narrower list that
# does not include bogons (e.g. firehol_abusers_1d), or connect via
# a public IP/domain with hairpin NAT on your router. See README for
# details.
urls = [ urls = [
"https://iplists.firehol.org/files/firehol_level1.netset", "https://iplists.firehol.org/files/firehol_level1.netset",
# "/local.file" # "/local.file"
@@ -378,11 +359,3 @@ bind-to = "127.0.0.1:3129"
http-path = "/" http-path = "/"
# prefix for metrics for prometheus # prefix for metrics for prometheus
metric-prefix = "mtg" metric-prefix = "mtg"
[dc]
dc1 = [
"149.154.175.50:443",
"149.154.175.54:443",
"149.154.175.59:443",
"[2001:b28:f23d:f001::a]:443"
]
+2 -2
View File
@@ -4,7 +4,7 @@ go 1.26
require ( require (
github.com/OneOfOne/xxhash v1.2.8 github.com/OneOfOne/xxhash v1.2.8
github.com/alecthomas/kong v1.15.0 github.com/alecthomas/kong v1.14.0
github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5
github.com/babolivier/go-doh-client v0.0.0-20201028162107-a76cff4cb8b6 github.com/babolivier/go-doh-client v0.0.0-20201028162107-a76cff4cb8b6
@@ -28,7 +28,7 @@ require (
require ( require (
github.com/beevik/ntp v1.5.0 github.com/beevik/ntp v1.5.0
github.com/ncruces/go-dns v1.3.3 github.com/ncruces/go-dns v1.3.2
github.com/pelletier/go-toml/v2 v2.3.0 github.com/pelletier/go-toml/v2 v2.3.0
github.com/pires/go-proxyproto v0.11.0 github.com/pires/go-proxyproto v0.11.0
github.com/things-go/go-socks5 v0.1.0 github.com/things-go/go-socks5 v0.1.0
+4 -4
View File
@@ -2,8 +2,8 @@ github.com/OneOfOne/xxhash v1.2.8 h1:31czK/TI9sNkxIKfaUfGlU47BAxQ0ztGgd9vPyqimf8
github.com/OneOfOne/xxhash v1.2.8/go.mod h1:eZbhyaAYD41SGSSsnmcpxVoRiQ/MPUTjUdIIOT9Um7Q= github.com/OneOfOne/xxhash v1.2.8/go.mod h1:eZbhyaAYD41SGSSsnmcpxVoRiQ/MPUTjUdIIOT9Um7Q=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0= github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k= github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/kong v1.15.0 h1:BVJstKbpO73zKpmIu+m/aLRrNmWwxXPIGTNin9VmLVI= github.com/alecthomas/kong v1.14.0 h1:gFgEUZWu2ZmZ+UhyZ1bDhuutbKN1nTtJTwh19Wsn21s=
github.com/alecthomas/kong v1.15.0/go.mod h1:wrlbXem1CWqUV5Vbmss5ISYhsVPkBb1Yo7YKJghju2I= github.com/alecthomas/kong v1.14.0/go.mod h1:wrlbXem1CWqUV5Vbmss5ISYhsVPkBb1Yo7YKJghju2I=
github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs= github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4= github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vSQ6PWWSL9lK8qwHozUj03+zLoEB8O0= github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vSQ6PWWSL9lK8qwHozUj03+zLoEB8O0=
@@ -48,8 +48,8 @@ github.com/miekg/dns v1.1.51 h1:0+Xg7vObnhrz/4ZCZcZh7zPXlmU0aveS2HDBd0m0qSo=
github.com/miekg/dns v1.1.51/go.mod h1:2Z9d3CP1LQWihRZUf29mQ19yDThaI4DAYzte2CaQW5c= github.com/miekg/dns v1.1.51/go.mod h1:2Z9d3CP1LQWihRZUf29mQ19yDThaI4DAYzte2CaQW5c=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/ncruces/go-dns v1.3.3 h1:59OV7XoJrTCoUMZjWRVs4GOjtntMTZqiQ5Mn+BT13hk= github.com/ncruces/go-dns v1.3.2 h1:kBLuUZBgkQ4qF4WDXZRQ4rG0Gk6sLVJQ5tESkWrxUa0=
github.com/ncruces/go-dns v1.3.3/go.mod h1:tuzixNY8PY/M7yUzcvRbUaeLs3ifIdydpi5H2bfRU+s= github.com/ncruces/go-dns v1.3.2/go.mod h1:tuzixNY8PY/M7yUzcvRbUaeLs3ifIdydpi5H2bfRU+s=
github.com/panjf2000/ants/v2 v2.12.0 h1:u9JhESo83i/GkZnhfTNuFMMWcNt7mnV1bGJ6FT4wXH8= github.com/panjf2000/ants/v2 v2.12.0 h1:u9JhESo83i/GkZnhfTNuFMMWcNt7mnV1bGJ6FT4wXH8=
github.com/panjf2000/ants/v2 v2.12.0/go.mod h1:tSQuaNQ6r6NRhPt+IZVUevvDyFMTs+eS4ztZc52uJTY= github.com/panjf2000/ants/v2 v2.12.0/go.mod h1:tSQuaNQ6r6NRhPt+IZVUevvDyFMTs+eS4ztZc52uJTY=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc= github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
-19
View File
@@ -50,12 +50,6 @@ var (
tplEDCConnect = template.Must( tplEDCConnect = template.Must(
template.New("").Parse(" ❌ DC {{ .dc }}: {{ .error }}\n"), template.New("").Parse(" ❌ DC {{ .dc }}: {{ .error }}\n"),
) )
tplODCIPConnect = template.Must(
template.New("").Parse(" ✅ {{ .ip }}\n"),
)
tplEDCIPConnect = template.Must(
template.New("").Parse(" ❌ {{ .ip }}: {{ .error }}\n"),
)
tplODNSSNIMatch = template.Must( tplODNSSNIMatch = template.Must(
template.New("").Parse(" ✅ IP address {{ .ip }} matches secret hostname {{ .hostname }}\n"), template.New("").Parse(" ✅ IP address {{ .ip }} matches secret hostname {{ .hostname }}\n"),
@@ -103,12 +97,6 @@ func (d *Doctor) Run(cli *CLI, version string) error {
conf.Network.Timeout.TCP.Get(10*time.Second), conf.Network.Timeout.TCP.Get(10*time.Second),
conf.Network.Timeout.HTTP.Get(0), conf.Network.Timeout.HTTP.Get(0),
conf.Network.Timeout.Idle.Get(0), conf.Network.Timeout.Idle.Get(0),
net.KeepAliveConfig{
Enable: !conf.Network.KeepAlive.Disabled.Get(false),
Idle: conf.Network.KeepAlive.Idle.Get(0),
Interval: conf.Network.KeepAlive.Interval.Get(0),
Count: int(conf.Network.KeepAlive.Count.Get(0)),
},
) )
fmt.Println("Validate native network connectivity") fmt.Println("Validate native network connectivity")
@@ -291,15 +279,8 @@ func (d *Doctor) checkNetworkAddresses(ntw mtglib.Network, addresses []string) e
for _, addr := range checkAddresses { for _, addr := range checkAddresses {
conn, err = ntw.DialContext(ctx, "tcp", addr) conn, err = ntw.DialContext(ctx, "tcp", addr)
if err != nil { if err != nil {
tplEDCIPConnect.Execute(os.Stdout, map[string]any{
"error": err,
"ip": addr,
})
continue continue
} }
tplODCIPConnect.Execute(os.Stdout, map[string]any{
"ip": addr,
})
conn.Close() //nolint: errcheck conn.Close() //nolint: errcheck
+2 -8
View File
@@ -5,6 +5,7 @@ import (
"fmt" "fmt"
"net" "net"
"os" "os"
"time"
"github.com/9seconds/mtg/v2/antireplay" "github.com/9seconds/mtg/v2/antireplay"
"github.com/9seconds/mtg/v2/events" "github.com/9seconds/mtg/v2/events"
@@ -50,12 +51,6 @@ func makeNetwork(conf *config.Config, version string) (mtglib.Network, error) {
conf.Network.Timeout.TCP.Get(0), conf.Network.Timeout.TCP.Get(0),
conf.Network.Timeout.HTTP.Get(0), conf.Network.Timeout.HTTP.Get(0),
conf.Network.Timeout.Idle.Get(0), conf.Network.Timeout.Idle.Get(0),
net.KeepAliveConfig{
Enable: !conf.Network.KeepAlive.Disabled.Get(false),
Idle: conf.Network.KeepAlive.Idle.Get(0),
Interval: conf.Network.KeepAlive.Interval.Get(0),
Count: int(conf.Network.KeepAlive.Count.Get(0)),
},
) )
proxyDialers := make([]mtglib.Network, len(conf.Network.Proxies)) proxyDialers := make([]mtglib.Network, len(conf.Network.Proxies))
@@ -268,8 +263,7 @@ func runProxy(conf *config.Config, version string) error { //nolint: funlen
AllowFallbackOnUnknownDC: conf.AllowFallbackOnUnknownDC.Get(false), AllowFallbackOnUnknownDC: conf.AllowFallbackOnUnknownDC.Get(false),
TolerateTimeSkewness: conf.TolerateTimeSkewness.Value, TolerateTimeSkewness: conf.TolerateTimeSkewness.Value,
IdleTimeout: conf.Network.Timeout.Idle.Get(mtglib.DefaultIdleTimeout), IdleTimeout: conf.Network.Timeout.Idle.Get(time.Minute),
HandshakeTimeout: conf.Network.Timeout.Handshake.Get(mtglib.DefaultHandshakeTimeout),
DoppelGangerURLs: doppelGangerURLs, DoppelGangerURLs: doppelGangerURLs,
DoppelGangerPerRaid: conf.Defense.Doppelganger.Repeats.Get(mtglib.DoppelGangerPerRaid), DoppelGangerPerRaid: conf.Defense.Doppelganger.Repeats.Get(mtglib.DoppelGangerPerRaid),
+7 -25
View File
@@ -52,25 +52,18 @@ type Config struct {
Blocklist ListConfig `json:"blocklist"` Blocklist ListConfig `json:"blocklist"`
Allowlist ListConfig `json:"allowlist"` Allowlist ListConfig `json:"allowlist"`
Doppelganger struct { Doppelganger struct {
URLs []TypeHttpsURL `json:"urls"` URLs []TypeHttpsURL `json:"urls"`
Repeats TypeConcurrency `json:"repeats_per_raid"` Repeats TypeConcurrency `json:"repeats_per_raid"`
UpdateEach TypeDuration `json:"raid_each"` UpdateEach TypeDuration `json:"raid_each"`
DRS TypeBool `json:"drs"` DRS TypeBool `json:"drs"`
} `json:"doppelganger"` } `json:"doppelganger"`
} `json:"defense"` } `json:"defense"`
Network struct { Network struct {
Timeout struct { Timeout struct {
TCP TypeDuration `json:"tcp"` TCP TypeDuration `json:"tcp"`
HTTP TypeDuration `json:"http"` HTTP TypeDuration `json:"http"`
Idle TypeDuration `json:"idle"` Idle TypeDuration `json:"idle"`
Handshake TypeDuration `json:"handshake"`
} `json:"timeout"` } `json:"timeout"`
KeepAlive struct {
Disabled TypeBool `json:"disabled"`
Idle TypeDuration `json:"idle"`
Interval TypeDuration `json:"interval"`
Count TypeConcurrency `json:"count"`
} `json:"keepAlive"`
DOHIP TypeIP `json:"dohIp"` DOHIP TypeIP `json:"dohIp"`
DNS TypeDNSURI `json:"dns"` DNS TypeDNSURI `json:"dns"`
Proxies []TypeProxyURL `json:"proxies"` Proxies []TypeProxyURL `json:"proxies"`
@@ -91,17 +84,6 @@ type Config struct {
MetricPrefix TypeMetricPrefix `json:"metricPrefix"` MetricPrefix TypeMetricPrefix `json:"metricPrefix"`
} `json:"prometheus"` } `json:"prometheus"`
} `json:"stats"` } `json:"stats"`
DCs struct{
DC1 []TypeHostPort `json:"dc1"`
DC2 []TypeHostPort `json:"dc2"`
DC3 []TypeHostPort `json:"dc3"`
DC4 []TypeHostPort `json:"dc4"`
DC5 []TypeHostPort `json:"dc5"`
DC203 []TypeHostPort `json:"dc203"`
DC10001 []TypeHostPort `json:"dc10001"`
DC10002[]TypeHostPort `json:"dc10002"`
DC10003 []TypeHostPort `json:"dc10003"`
} `json:"dc"`
} }
func (c *Config) GetConcurrency(defaultValue uint) uint { func (c *Config) GetConcurrency(defaultValue uint) uint {
+7 -25
View File
@@ -47,25 +47,18 @@ type tomlConfig struct {
UpdateEach string `toml:"update-each" json:"updateEach,omitempty"` UpdateEach string `toml:"update-each" json:"updateEach,omitempty"`
} `toml:"allowlist" json:"allowlist,omitempty"` } `toml:"allowlist" json:"allowlist,omitempty"`
Doppelganger struct { Doppelganger struct {
URLs []string `toml:"urls" json:"urls,omitempty"` URLs []string `toml:"urls" json:"urls,omitempty"`
Repeats uint `toml:"repeats-per-raid" json:"repeats_per_raid,omitempty"` Repeats uint `toml:"repeats-per-raid" json:"repeats_per_raid,omitempty"`
UpdateEach string `toml:"raid-each" json:"raid_each,omitempty"` UpdateEach string `toml:"raid-each" json:"raid_each,omitempty"`
DRS bool `toml:"drs" json:"drs,omitempty"` DRS bool `toml:"drs" json:"drs,omitempty"`
} `toml:"doppelganger" json:"doppelganger,omitempty"` } `toml:"doppelganger" json:"doppelganger,omitempty"`
} `toml:"defense" json:"defense,omitempty"` } `toml:"defense" json:"defense,omitempty"`
Network struct { Network struct {
Timeout struct { Timeout struct {
TCP string `toml:"tcp" json:"tcp,omitempty"` TCP string `toml:"tcp" json:"tcp,omitempty"`
HTTP string `toml:"http" json:"http,omitempty"` HTTP string `toml:"http" json:"http,omitempty"`
Idle string `toml:"idle" json:"idle,omitempty"` Idle string `toml:"idle" json:"idle,omitempty"`
Handshake string `toml:"handshake" json:"handshake,omitempty"`
} `toml:"timeout" json:"timeout,omitempty"` } `toml:"timeout" json:"timeout,omitempty"`
KeepAlive struct {
Disabled bool `toml:"disabled" json:"disabled,omitempty"`
Idle string `toml:"idle" json:"idle,omitempty"`
Interval string `toml:"interval" json:"interval,omitempty"`
Count uint `toml:"count" json:"count,omitempty"`
} `toml:"keep-alive" json:"keepAlive,omitempty"`
DOHIP string `toml:"doh-ip" json:"dohIp,omitempty"` DOHIP string `toml:"doh-ip" json:"dohIp,omitempty"`
DNS string `toml:"dns" json:"dns,omitempty"` DNS string `toml:"dns" json:"dns,omitempty"`
Proxies []string `toml:"proxies" json:"proxies,omitempty"` Proxies []string `toml:"proxies" json:"proxies,omitempty"`
@@ -84,17 +77,6 @@ type tomlConfig struct {
MetricPrefix string `toml:"metric-prefix" json:"metricPrefix,omitempty"` MetricPrefix string `toml:"metric-prefix" json:"metricPrefix,omitempty"`
} `toml:"prometheus" json:"prometheus,omitempty"` } `toml:"prometheus" json:"prometheus,omitempty"`
} `toml:"stats" json:"stats,omitempty"` } `toml:"stats" json:"stats,omitempty"`
DCs struct{
DC1 []string `toml:"dc1" json:"dc1,omitempty"`
DC2 []string `toml:"dc2" json:"dc2,omitempty"`
DC3 []string `toml:"dc3" json:"dc3,omitempty"`
DC4 []string `toml:"dc4" json:"dc4,omitempty"`
DC5 []string `toml:"dc5" json:"dc5,omitempty"`
DC203 []string `toml:"dc203" json:"dc203,omitempty"`
DC10001 []string `toml:"dc10001" json:"dc10001,omitempty"`
DC10002 []string `toml:"dc10002" json:"dc10002,omitempty"`
DC10003 []string `toml:"dc10003" json:"dc10003,omitempty"`
} `toml:"dc" json:"dc,omitempty"`
} }
func Parse(rawData []byte) (*Config, error) { func Parse(rawData []byte) (*Config, error) {
-70
View File
@@ -4,77 +4,9 @@ import (
"fmt" "fmt"
"os" "os"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/internal/config" "github.com/9seconds/mtg/v2/internal/config"
) )
func readCustomDC(cfg *config.Config) {
if len(cfg.DCs.DC1) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC1 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[1] = ips
}
if len(cfg.DCs.DC2) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC2 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[2] = ips
}
if len(cfg.DCs.DC3) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC3 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[3] = ips
}
if len(cfg.DCs.DC4) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC4 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[4] = ips
}
if len(cfg.DCs.DC5) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC5 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[5] = ips
}
if len(cfg.DCs.DC203) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC203 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[203] = ips
}
if len(cfg.DCs.DC10001) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC10001 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[10001] = ips
}
if len(cfg.DCs.DC10002) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC10002 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[10002] = ips
}
if len(cfg.DCs.DC10003) > 0 {
var ips []string
for _, addr := range cfg.DCs.DC10003 {
ips = append(ips, addr.Value)
}
essentials.TelegramCoreAddresses[10003] = ips
}
}
func ReadConfig(path string) (*config.Config, error) { func ReadConfig(path string) (*config.Config, error) {
content, err := os.ReadFile(path) content, err := os.ReadFile(path)
if err != nil { if err != nil {
@@ -90,7 +22,5 @@ func ReadConfig(path string) (*config.Config, error) {
return nil, fmt.Errorf("invalid config: %w", err) return nil, fmt.Errorf("invalid config: %w", err)
} }
readCustomDC(conf)
return conf, nil return conf, nil
} }
+16 -16
View File
@@ -82,40 +82,40 @@ checksum = "sha256:4932cfca5e75bf60fe1c576edf459e5e809e6644664a068185d64b84af3fa
url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-windows-amd64.zip" url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-windows-amd64.zip"
[[tools.goreleaser]] [[tools.goreleaser]]
version = "2.15.2" version = "2.14.3"
backend = "aqua:goreleaser/goreleaser" backend = "aqua:goreleaser/goreleaser"
[tools.goreleaser."platforms.linux-arm64"] [tools.goreleaser."platforms.linux-arm64"]
checksum = "sha256:5db66761a98f6693161e49e1a95d28d2673a892ba60cb4a5e16736cafd41c4c9" checksum = "sha256:581a10e53c1176b3e81ee45cf531e02dbf899db0bc7b795669347df4276ce948"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_arm64.tar.gz" url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Linux_arm64.tar.gz"
provenance = "cosign" provenance = "cosign"
[tools.goreleaser."platforms.linux-arm64-musl"] [tools.goreleaser."platforms.linux-arm64-musl"]
checksum = "sha256:5db66761a98f6693161e49e1a95d28d2673a892ba60cb4a5e16736cafd41c4c9" checksum = "sha256:581a10e53c1176b3e81ee45cf531e02dbf899db0bc7b795669347df4276ce948"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_arm64.tar.gz" url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Linux_arm64.tar.gz"
provenance = "cosign" provenance = "cosign"
[tools.goreleaser."platforms.linux-x64"] [tools.goreleaser."platforms.linux-x64"]
checksum = "sha256:0ebdbf0353aba566b969dde746cc4e4806f96c27aa2f3971b229a9df7611fedc" checksum = "sha256:dc7faeeeb6da8bdfda788626263a4ae725892a8c7504b975c3234127d4a44579"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_x86_64.tar.gz" url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Linux_x86_64.tar.gz"
provenance = "cosign" provenance = "cosign"
[tools.goreleaser."platforms.linux-x64-musl"] [tools.goreleaser."platforms.linux-x64-musl"]
checksum = "sha256:0ebdbf0353aba566b969dde746cc4e4806f96c27aa2f3971b229a9df7611fedc" checksum = "sha256:dc7faeeeb6da8bdfda788626263a4ae725892a8c7504b975c3234127d4a44579"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_x86_64.tar.gz" url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Linux_x86_64.tar.gz"
provenance = "cosign" provenance = "cosign"
[tools.goreleaser."platforms.macos-arm64"] [tools.goreleaser."platforms.macos-arm64"]
checksum = "sha256:0e6bd67688ac949780bf1166813a91f89856898ef4c40d7d46c2c74ebaa4b9ee" checksum = "sha256:3507798489e107a78aff36b169de48148a335ac26eb3161608d905f3f3a957bd"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Darwin_all.tar.gz" url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Darwin_all.tar.gz"
provenance = "github-attestations" provenance = "cosign"
[tools.goreleaser."platforms.macos-x64"] [tools.goreleaser."platforms.macos-x64"]
checksum = "sha256:0e6bd67688ac949780bf1166813a91f89856898ef4c40d7d46c2c74ebaa4b9ee" checksum = "sha256:3507798489e107a78aff36b169de48148a335ac26eb3161608d905f3f3a957bd"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Darwin_all.tar.gz" url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Darwin_all.tar.gz"
provenance = "cosign" provenance = "cosign"
[tools.goreleaser."platforms.windows-x64"] [tools.goreleaser."platforms.windows-x64"]
checksum = "sha256:7459832946dbe122c144f8d7f87484d8572ca005b779310aa6bb03346e8de17a" checksum = "sha256:3deea8ff471aa258a2d99f3e5302971d7028647ae8ddaf103257a8113e485a31"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Windows_x86_64.zip" url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.3/goreleaser_Windows_x86_64.zip"
provenance = "cosign" provenance = "cosign"
+5 -50
View File
@@ -3,11 +3,9 @@ package mtglib
import ( import (
"bytes" "bytes"
"context" "context"
"errors"
"fmt" "fmt"
"io" "io"
"net" "net"
"sync/atomic"
"time" "time"
"github.com/9seconds/mtg/v2/essentials" "github.com/9seconds/mtg/v2/essentials"
@@ -99,63 +97,20 @@ func newConnProxyProtocol(source, target essentials.Conn) *connProxyProtocol {
} }
} }
// idleTracker is a shared idle tracker for a pair of relay connections.
// Both directions update the same timestamp so that activity in one direction
// prevents the other (idle) direction from timing out.
type idleTracker struct {
lastActive atomic.Pointer[time.Time]
timeout time.Duration
}
func newIdleTracker(timeout time.Duration) *idleTracker {
t := &idleTracker{timeout: timeout}
t.touch()
return t
}
func (t *idleTracker) touch() {
stamp := time.Now()
t.lastActive.Store(&stamp)
}
func (t *idleTracker) isIdle() bool {
return time.Since(*t.lastActive.Load()) >= t.timeout
}
type connIdleTimeout struct { type connIdleTimeout struct {
essentials.Conn essentials.Conn
tracker *idleTracker timeout time.Duration
} }
func (c connIdleTimeout) Read(b []byte) (int, error) { func (c connIdleTimeout) Read(b []byte) (int, error) {
var netErr net.Error c.SetReadDeadline(time.Now().Add(c.timeout)) //nolint: errcheck
for { return c.Conn.Read(b) //nolint: wrapcheck
c.SetReadDeadline(time.Now().Add(c.tracker.timeout)) //nolint: errcheck
n, err := c.Conn.Read(b)
switch {
case err == nil:
c.tracker.touch()
return n, nil
case errors.As(err, &netErr) && netErr.Timeout() && !c.tracker.isIdle():
continue
}
return n, err
}
} }
func (c connIdleTimeout) Write(b []byte) (int, error) { func (c connIdleTimeout) Write(b []byte) (int, error) {
c.SetWriteDeadline(time.Now().Add(c.tracker.timeout)) //nolint: errcheck c.SetWriteDeadline(time.Now().Add(c.timeout)) //nolint: errcheck
n, err := c.Conn.Write(b) return c.Conn.Write(b) //nolint: wrapcheck
if n > 0 {
c.tracker.touch()
}
return n, err //nolint: wrapcheck
} }
-151
View File
@@ -16,12 +16,6 @@ import (
"github.com/stretchr/testify/suite" "github.com/stretchr/testify/suite"
) )
type netTimeoutError struct{}
func (e netTimeoutError) Error() string { return "i/o timeout" }
func (e netTimeoutError) Timeout() bool { return true }
func (e netTimeoutError) Temporary() bool { return true }
type ConnRewindBaseConn struct { type ConnRewindBaseConn struct {
testlib.EssentialsConnMock testlib.EssentialsConnMock
@@ -297,141 +291,6 @@ func (suite *ConnProxyProtocolTestSuite) TearDownTest() {
suite.targetConnMock.AssertExpectations(suite.T()) suite.targetConnMock.AssertExpectations(suite.T())
} }
type IdleTrackerTestSuite struct {
suite.Suite
}
func (suite *IdleTrackerTestSuite) TestNewNotIdle() {
tracker := newIdleTracker(time.Second)
suite.False(tracker.isIdle())
}
func (suite *IdleTrackerTestSuite) TestIdleAfterTimeout() {
tracker := newIdleTracker(10 * time.Millisecond)
time.Sleep(20 * time.Millisecond)
suite.True(tracker.isIdle())
}
func (suite *IdleTrackerTestSuite) TestTouchResetsIdle() {
tracker := newIdleTracker(50 * time.Millisecond)
time.Sleep(30 * time.Millisecond)
tracker.touch()
suite.False(tracker.isIdle())
}
type ConnIdleTimeoutTestSuite struct {
suite.Suite
connMock *testlib.EssentialsConnMock
tracker *idleTracker
conn connIdleTimeout
}
func (suite *ConnIdleTimeoutTestSuite) SetupTest() {
suite.connMock = &testlib.EssentialsConnMock{}
suite.tracker = newIdleTracker(time.Second)
suite.conn = connIdleTimeout{
Conn: suite.connMock,
tracker: suite.tracker,
}
}
func (suite *ConnIdleTimeoutTestSuite) TearDownTest() {
suite.connMock.AssertExpectations(suite.T())
}
func (suite *ConnIdleTimeoutTestSuite) TestReadOk() {
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(5, nil)
n, err := suite.conn.Read(make([]byte, 10))
suite.NoError(err)
suite.Equal(5, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestReadNonTimeoutErr() {
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, io.EOF)
n, err := suite.conn.Read(make([]byte, 10))
suite.True(errors.Is(err, io.EOF))
suite.Equal(0, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestReadTimeoutRetriesWhenNotIdle() {
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, netTimeoutError{})
suite.connMock.On("Read", mock.Anything).Once().Return(5, nil)
n, err := suite.conn.Read(make([]byte, 10))
suite.NoError(err)
suite.Equal(5, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestReadTimeoutClosesWhenIdle() {
suite.tracker = newIdleTracker(time.Millisecond)
suite.conn = connIdleTimeout{
Conn: suite.connMock,
tracker: suite.tracker,
}
time.Sleep(5 * time.Millisecond)
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, netTimeoutError{})
n, err := suite.conn.Read(make([]byte, 10))
suite.Equal(0, n)
netErr, ok := err.(net.Error) //nolint: errorlint
suite.True(ok)
suite.True(netErr.Timeout())
}
func (suite *ConnIdleTimeoutTestSuite) TestSharedTrackerPreventsFalseTimeout() {
connMock2 := &testlib.EssentialsConnMock{}
conn2 := connIdleTimeout{
Conn: connMock2,
tracker: suite.tracker,
}
connMock2.On("SetWriteDeadline", mock.Anything).Return(nil)
connMock2.On("Write", mock.Anything).Once().Return(5, nil)
_, _ = conn2.Write(make([]byte, 5))
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, netTimeoutError{})
suite.connMock.On("Read", mock.Anything).Once().Return(3, nil)
n, err := suite.conn.Read(make([]byte, 10))
suite.NoError(err)
suite.Equal(3, n)
connMock2.AssertExpectations(suite.T())
}
func (suite *ConnIdleTimeoutTestSuite) TestWriteOk() {
suite.connMock.On("SetWriteDeadline", mock.Anything).Return(nil)
suite.connMock.On("Write", mock.Anything).Once().Return(5, nil)
n, err := suite.conn.Write(make([]byte, 5))
suite.NoError(err)
suite.Equal(5, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestWriteErr() {
suite.connMock.On("SetWriteDeadline", mock.Anything).Return(nil)
suite.connMock.On("Write", mock.Anything).Once().Return(0, io.EOF)
n, err := suite.conn.Write(make([]byte, 5))
suite.True(errors.Is(err, io.EOF))
suite.Equal(0, n)
}
func TestConnTraffic(t *testing.T) { func TestConnTraffic(t *testing.T) {
t.Parallel() t.Parallel()
suite.Run(t, &ConnTrafficTestSuite{}) suite.Run(t, &ConnTrafficTestSuite{})
@@ -446,13 +305,3 @@ func TestConnProxyProtocol(t *testing.T) {
t.Parallel() t.Parallel()
suite.Run(t, &ConnProxyProtocolTestSuite{}) suite.Run(t, &ConnProxyProtocolTestSuite{})
} }
func TestIdleTracker(t *testing.T) {
t.Parallel()
suite.Run(t, &IdleTrackerTestSuite{})
}
func TestConnIdleTimeout(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnIdleTimeoutTestSuite{})
}
+2 -7
View File
@@ -77,13 +77,8 @@ const (
// DefaultIdleTimeout is a default timeout for closing a connection in case of // DefaultIdleTimeout is a default timeout for closing a connection in case of
// idling. // idling.
// //
// Set to 5 minutes to survive typical mobile sleep periods (2-5 min) and // Deprecated: no longer in use because of changed TCP relay algorithm.
// avoid racing with MTProto ping_delay_disconnect (~60s interval). DefaultIdleTimeout = time.Minute
DefaultIdleTimeout = 5 * time.Minute
// DefaultHandshakeTimeout defines a time period during which the
// all handshake ceremonies must be completed.
DefaultHandshakeTimeout = 10 * time.Second
// DefaultTolerateTimeSkewness is a default timeout for time skewness on a // DefaultTolerateTimeSkewness is a default timeout for time skewness on a
// faketls timeout verification. // faketls timeout verification.
+2 -6
View File
@@ -5,19 +5,15 @@ type dcView struct {
} }
func (d dcView) getV4(dc int) []Addr { func (d dcView) getV4(dc int) []Addr {
var addrs []Addr addrs := d.publicConfigs.getV4(dc)
addrs = append(addrs, defaultDCAddrSet.getV4(dc)...) addrs = append(addrs, defaultDCAddrSet.getV4(dc)...)
addrs = append(addrs, d.publicConfigs.getV4(dc)...)
return addrs return addrs
} }
func (d dcView) getV6(dc int) []Addr { func (d dcView) getV6(dc int) []Addr {
var addrs []Addr addrs := d.publicConfigs.getV6(dc)
addrs = append(addrs, defaultDCAddrSet.getV6(dc)...) addrs = append(addrs, defaultDCAddrSet.getV6(dc)...)
addrs = append(addrs, d.publicConfigs.getV6(dc)...)
return addrs return addrs
} }
+6 -12
View File
@@ -61,29 +61,23 @@ func (s Scout) learn(ctx context.Context, url string) (ScoutResult, error) {
client.CloseIdleConnections() client.CloseIdleConnections()
} }
if err != nil { if err != nil || len(results.data) == 0 {
return ScoutResult{}, err return ScoutResult{}, err
} }
data, writeIndex := results.Snapshot()
if len(data) == 0 {
return ScoutResult{}, nil
}
var result ScoutResult var result ScoutResult
// Compute inter-record durations (existing logic). // Compute inter-record durations (existing logic).
lastTimestamp := time.Time{} lastTimestamp := time.Time{}
for i, v := range data { for i, v := range results.data {
if v.recordType != tls.TypeApplicationData { if v.recordType != tls.TypeApplicationData {
continue continue
} }
if lastTimestamp.IsZero() { if lastTimestamp.IsZero() {
if i > 0 { if i > 0 {
lastTimestamp = data[i-1].timestamp lastTimestamp = results.data[i-1].timestamp
} else { } else {
lastTimestamp = v.timestamp lastTimestamp = v.timestamp
} }
@@ -96,12 +90,12 @@ func (s Scout) learn(ctx context.Context, url string) (ScoutResult, error) {
// Compute cert size: sum of ApplicationData payload between CCS and // Compute cert size: sum of ApplicationData payload between CCS and
// the first client Write (which marks the end of server handshake). // the first client Write (which marks the end of server handshake).
seenCCS := false seenCCS := false
boundary := writeIndex boundary := results.writeIndex
if boundary < 0 { if boundary < 0 {
boundary = len(data) boundary = len(results.data)
} }
for i, v := range data { for i, v := range results.data {
if i >= boundary { if i >= boundary {
break break
} }
+1 -20
View File
@@ -1,10 +1,6 @@
package doppel package doppel
import ( import "time"
"slices"
"sync"
"time"
)
const ( const (
ScoutConnCollectedPreallocSize = 100 ScoutConnCollectedPreallocSize = 100
@@ -17,38 +13,23 @@ type ScoutConnResult struct {
} }
type ScoutConnCollected struct { type ScoutConnCollected struct {
mu sync.Mutex
data []ScoutConnResult data []ScoutConnResult
writeIndex int // index at which client first wrote post-handshake data; -1 if not set writeIndex int // index at which client first wrote post-handshake data; -1 if not set
} }
func (s *ScoutConnCollected) Add(record byte, payloadLen int) { func (s *ScoutConnCollected) Add(record byte, payloadLen int) {
s.mu.Lock()
s.data = append(s.data, ScoutConnResult{ s.data = append(s.data, ScoutConnResult{
timestamp: time.Now(), timestamp: time.Now(),
recordType: record, recordType: record,
payloadLen: payloadLen, payloadLen: payloadLen,
}) })
s.mu.Unlock()
} }
// MarkWrite records the current data length as the handshake boundary. // MarkWrite records the current data length as the handshake boundary.
func (s *ScoutConnCollected) MarkWrite() { func (s *ScoutConnCollected) MarkWrite() {
s.mu.Lock()
if s.writeIndex < 0 { if s.writeIndex < 0 {
s.writeIndex = len(s.data) s.writeIndex = len(s.data)
} }
s.mu.Unlock()
}
// Snapshot returns a copy of the collected data and the write index.
func (s *ScoutConnCollected) Snapshot() ([]ScoutConnResult, int) {
s.mu.Lock()
snapshot := slices.Clone(s.data)
writeIndex := s.writeIndex
s.mu.Unlock()
return snapshot, writeIndex
} }
func NewScoutConnCollected() *ScoutConnCollected { func NewScoutConnCollected() *ScoutConnCollected {
@@ -1,7 +1,6 @@
package doppel package doppel
import ( import (
"sync"
"testing" "testing"
"time" "time"
@@ -17,10 +16,8 @@ func (suite *ScoutConnCollectedTestSuite) TestAddSingle() {
collected := NewScoutConnCollected() collected := NewScoutConnCollected()
collected.Add(tls.TypeApplicationData, 100) collected.Add(tls.TypeApplicationData, 100)
data, _ := collected.Snapshot() suite.Len(collected.data, 1)
suite.Equal(byte(tls.TypeApplicationData), collected.data[0].recordType)
suite.Len(data, 1)
suite.Equal(byte(tls.TypeApplicationData), data[0].recordType)
} }
func (suite *ScoutConnCollectedTestSuite) TestAddTimestampsAreMonotonic() { func (suite *ScoutConnCollectedTestSuite) TestAddTimestampsAreMonotonic() {
@@ -34,52 +31,11 @@ func (suite *ScoutConnCollectedTestSuite) TestAddTimestampsAreMonotonic() {
time.Sleep(time.Microsecond) time.Sleep(time.Microsecond)
collected.Add(tls.TypeApplicationData, 100) collected.Add(tls.TypeApplicationData, 100)
data, _ := collected.Snapshot() for i := 1; i < len(collected.data); i++ {
suite.True(collected.data[i].timestamp.After(collected.data[i-1].timestamp))
for i := 1; i < len(data); i++ {
suite.True(data[i].timestamp.After(data[i-1].timestamp))
} }
} }
func (suite *ScoutConnCollectedTestSuite) TestConcurrentAddSnapshot() {
collected := NewScoutConnCollected()
var wg sync.WaitGroup
wg.Add(3)
go func() {
defer wg.Done()
for i := 0; i < 1000; i++ {
collected.Add(tls.TypeApplicationData, i)
}
}()
go func() {
defer wg.Done()
for i := 0; i < 100; i++ {
collected.MarkWrite()
}
}()
go func() {
defer wg.Done()
for i := 0; i < 1000; i++ {
// call Snapshot concurrently to exercise the lock under -race
collected.Snapshot() //nolint:errcheck
}
}()
wg.Wait()
data, writeIndex := collected.Snapshot()
suite.Len(data, 1000)
suite.GreaterOrEqual(writeIndex, 0)
}
func TestScoutConnCollected(t *testing.T) { func TestScoutConnCollected(t *testing.T) {
t.Parallel() t.Parallel()
suite.Run(t, &ScoutConnCollectedTestSuite{}) suite.Run(t, &ScoutConnCollectedTestSuite{})
+82 -34
View File
@@ -6,12 +6,13 @@ import (
"crypto/sha256" "crypto/sha256"
"crypto/subtle" "crypto/subtle"
"encoding/binary" "encoding/binary"
"errors"
"fmt" "fmt"
"io" "io"
"net" "net"
"slices" "slices"
"time" "time"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
) )
const ( const (
@@ -21,19 +22,12 @@ const (
// record_type(1) + version(2) + size(2) + handshake_type(1) + uint24_length(3) + client_version(2) // record_type(1) + version(2) + size(2) + handshake_type(1) + uint24_length(3) + client_version(2)
RandomOffset = 1 + 2 + 2 + 1 + 3 + 2 RandomOffset = 1 + 2 + 2 + 1 + 3 + 2
// https://datatracker.ietf.org/doc/html/rfc8701#name-grease-values
// https://medium.com/asecuritysite-when-bob-met-alice/in-cybersecurity-what-is-grease-9f8850558dea
GreaseMask = 0x0f0f
GreaseValueType = 0x0a0a
sniDNSNamesListType = 0 sniDNSNamesListType = 0
) )
var ( var (
emptyRandom = [RandomLen]byte{} emptyRandom = [RandomLen]byte{}
extTypeSNI = [2]byte{} extTypeSNI = [2]byte{}
ErrCannotFindCipher = errors.New("cannot find a cipher")
) )
type ClientHello struct { type ClientHello struct {
@@ -48,6 +42,11 @@ func ReadClientHello(
hostname string, hostname string,
tolerateTimeSkewness time.Duration, tolerateTimeSkewness time.Duration,
) (*ClientHello, error) { ) (*ClientHello, error) {
if err := conn.SetReadDeadline(time.Now().Add(ClientHelloReadTimeout)); err != nil {
return nil, fmt.Errorf("cannot set read deadline: %w", err)
}
defer conn.SetReadDeadline(resetDeadline) //nolint: errcheck
// This is how FakeTLS is organized: // This is how FakeTLS is organized:
// 1. We create sha256 HMAC with a given secret // 1. We create sha256 HMAC with a given secret
// 2. We dump there a whole TLS frame except of the fact that random // 2. We dump there a whole TLS frame except of the fact that random
@@ -57,17 +56,25 @@ func ReadClientHello(
// 4. New digest should be all 0 except of last 4 bytes // 4. New digest should be all 0 except of last 4 bytes
// 5. Last 4 bytes are little endian uint32 of UNIX timestamp when // 5. Last 4 bytes are little endian uint32 of UNIX timestamp when
// this message was created. // this message was created.
clientHelloCopy, handshakeReader, err := parseClientHello(conn) handshakeCopyBuf := &bytes.Buffer{}
reader := io.TeeReader(conn, handshakeCopyBuf)
reader, err := parseTLSHeader(reader)
if err != nil { if err != nil {
return nil, fmt.Errorf("cannot read client hello: %w", err) return nil, fmt.Errorf("cannot parse tls header: %w", err)
} }
hello, err := parseHandshake(handshakeReader) reader, err = parseHandshakeHeader(reader)
if err != nil {
return nil, fmt.Errorf("cannot parse handshake header: %w", err)
}
hello, err := parseHandshake(reader)
if err != nil { if err != nil {
return nil, fmt.Errorf("cannot parse handshake: %w", err) return nil, fmt.Errorf("cannot parse handshake: %w", err)
} }
sniHostnames, err := parseSNI(handshakeReader) sniHostnames, err := parseSNI(reader)
if err != nil { if err != nil {
return nil, fmt.Errorf("cannot parse SNI: %w", err) return nil, fmt.Errorf("cannot parse SNI: %w", err)
} }
@@ -78,10 +85,10 @@ func ReadClientHello(
digest := hmac.New(sha256.New, secret) digest := hmac.New(sha256.New, secret)
// we write a copy of the handshake with client random all nullified. // we write a copy of the handshake with client random all nullified.
digest.Write(clientHelloCopy.Next(RandomOffset)) digest.Write(handshakeCopyBuf.Next(RandomOffset))
clientHelloCopy.Next(RandomLen) handshakeCopyBuf.Next(RandomLen)
digest.Write(emptyRandom[:]) digest.Write(emptyRandom[:])
digest.Write(clientHelloCopy.Bytes()) digest.Write(handshakeCopyBuf.Bytes())
computed := digest.Sum(nil) computed := digest.Sum(nil)
@@ -103,6 +110,58 @@ func ReadClientHello(
return hello, nil return hello, nil
} }
func parseTLSHeader(r io.Reader) (io.Reader, error) {
// record_type(1) + version(2) + size(2)
// 16 - type is 0x16 (handshake record)
// 03 01 - protocol version is "3,1" (also known as TLS 1.0)
// 00 f8 - 0xF8 (248) bytes of handshake message follows
header := [1 + 2 + 2]byte{}
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read record header: %w", err)
}
if header[0] != tls.TypeHandshake {
return nil, fmt.Errorf("unexpected record type %#x", header[0])
}
if header[1] != 3 || header[2] != 1 {
return nil, fmt.Errorf("unexpected protocol version %#x %#x", header[1], header[2])
}
length := int64(binary.BigEndian.Uint16(header[3:]))
buf := &bytes.Buffer{}
_, err := io.CopyN(buf, r, length)
return buf, err
}
func parseHandshakeHeader(r io.Reader) (io.Reader, error) {
// type(1) + size(3 / uint24)
// 01 - handshake message type 0x01 (client hello)
// 00 00 f4 - 0xF4 (244) bytes of client hello data follows
header := [1 + 3]byte{}
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read handshake header: %w", err)
}
if header[0] != TypeHandshakeClient {
return nil, fmt.Errorf("incorrect handshake type: %#x", header[0])
}
// unfortunately there is not uint24 in golang, so we just reust header
header[0] = 0
length := int64(binary.BigEndian.Uint32(header[:]))
buf := &bytes.Buffer{}
_, err := io.CopyN(buf, r, length)
return buf, err
}
func parseHandshake(r io.Reader) (*ClientHello, error) { func parseHandshake(r io.Reader) (*ClientHello, error) {
// A protocol version of "3,3" (meaning TLS 1.2) is given. // A protocol version of "3,3" (meaning TLS 1.2) is given.
header := [2]byte{} header := [2]byte{}
@@ -133,27 +192,16 @@ func parseHandshake(r io.Reader) (*ClientHello, error) {
cipherSuiteLen := int64(binary.BigEndian.Uint16(header[:])) cipherSuiteLen := int64(binary.BigEndian.Uint16(header[:]))
// Pick the first non-GREASE cipher suite from the list. // we do not care about picking up any cipher. we pick the first one,
// Real TLS servers never select GREASE values (RFC 8701, pattern 0x?a?a), // so it is always should be present.
// so echoing them back is a trivial DPI fingerprint. if _, err := io.ReadFull(r, header[:]); err != nil {
// cipherSuiteLen is in bytes; each cipher suite is 2 bytes. return nil, fmt.Errorf("cannot read first cipher suite: %w", err)
for range cipherSuiteLen / 2 {
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read cipher suite: %w", err)
}
if hello.CipherSuite != 0 {
// do not forget we have to scan until the end
continue
}
if cs := binary.BigEndian.Uint16(header[:]); cs&GreaseMask != GreaseValueType {
hello.CipherSuite = cs
}
} }
if hello.CipherSuite == 0 { hello.CipherSuite = binary.BigEndian.Uint16(header[:])
return nil, ErrCannotFindCipher
if _, err := io.CopyN(io.Discard, r, cipherSuiteLen-2); err != nil {
return nil, fmt.Errorf("cannot skip remaining cipher suites: %w", err)
} }
if _, err := io.ReadFull(r, header[:1]); err != nil { if _, err := io.ReadFull(r, header[:1]); err != nil {
@@ -12,6 +12,7 @@ import (
"github.com/9seconds/mtg/v2/mtglib" "github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/tls/fake" "github.com/9seconds/mtg/v2/mtglib/internal/tls/fake"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite" "github.com/stretchr/testify/suite"
) )
@@ -70,6 +71,11 @@ func (suite *ParseClientHelloSnapshotTestSuite) makeConn(data []byte) *parseClie
readBuf: readBuf, readBuf: readBuf,
} }
connMock.
On("SetReadDeadline", mock.AnythingOfType("time.Time")).
Twice().
Return(nil)
return connMock return connMock
} }
+23 -290
View File
@@ -2,11 +2,9 @@ package fake_test
import ( import (
"bytes" "bytes"
cryptotls "crypto/tls"
"encoding/binary" "encoding/binary"
"encoding/json" "errors"
"io" "io"
"os"
"testing" "testing"
"time" "time"
@@ -14,6 +12,7 @@ import (
"github.com/9seconds/mtg/v2/mtglib" "github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/tls" "github.com/9seconds/mtg/v2/mtglib/internal/tls"
"github.com/9seconds/mtg/v2/mtglib/internal/tls/fake" "github.com/9seconds/mtg/v2/mtglib/internal/tls/fake"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite" "github.com/stretchr/testify/suite"
) )
@@ -52,6 +51,11 @@ func (suite *ParseClientHelloTestSuite) SetupTest() {
suite.connMock = &parseClientHelloConnMock{ suite.connMock = &parseClientHelloConnMock{
readBuf: suite.readBuf, readBuf: suite.readBuf,
} }
suite.connMock.
On("SetReadDeadline", mock.AnythingOfType("time.Time")).
Twice().
Return(nil)
} }
func (suite *ParseClientHelloTestSuite) TearDownTest() { func (suite *ParseClientHelloTestSuite) TearDownTest() {
@@ -63,11 +67,23 @@ type ParseClientHello_TLSHeaderTestSuite struct {
} }
func (suite *ParseClientHello_TLSHeaderTestSuite) TestEmpty() { func (suite *ParseClientHello_TLSHeaderTestSuite) TestEmpty() {
suite.connMock.ExpectedCalls = []*mock.Call{}
suite.connMock.
On("SetReadDeadline", mock.AnythingOfType("time.Time")).
Once().
Return(errors.New("fail"))
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime) _, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read client hello") suite.ErrorContains(err, "fail")
} }
func (suite *ParseClientHello_TLSHeaderTestSuite) TestNothing() { func (suite *ParseClientHello_TLSHeaderTestSuite) TestNothing() {
suite.connMock.ExpectedCalls = []*mock.Call{}
suite.connMock.
On("SetReadDeadline", mock.AnythingOfType("time.Time")).
Twice().
Return(nil)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime) _, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorIs(err, io.EOF) suite.ErrorIs(err, io.EOF)
} }
@@ -216,13 +232,12 @@ func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadCipherSuiteLe
} }
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadFirstCipherSuite() { func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadFirstCipherSuite() {
body := make([]byte, 2+fake.RandomLen+1+2+1) // cipherSuiteLen=2 but only 1 byte available body := make([]byte, 2+fake.RandomLen+1+2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2)
suite.writeBody(body) suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime) _, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read cipher suite") suite.ErrorContains(err, "cannot read first cipher suite")
} }
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipRemainingCipherSuites() { func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipRemainingCipherSuites() {
@@ -232,27 +247,12 @@ func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipRemainingCiph
suite.writeBody(body) suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime) _, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read cipher suite") suite.ErrorContains(err, "cannot skip remaining cipher suites")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotFindCipher() {
// All cipher suites are GREASE values — must return ErrCannotFindCipher.
body := make([]byte, 2+fake.RandomLen+1+2+4+1)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 4)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2:], 0x0a0a)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2+2:], 0x1a1a)
body[2+fake.RandomLen+1+2+4] = 1
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorIs(err, fake.ErrCannotFindCipher)
} }
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadCompressionMethodsLength() { func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadCompressionMethodsLength() {
body := make([]byte, 2+fake.RandomLen+1+2+2) body := make([]byte, 2+fake.RandomLen+1+2+2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2) binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2:], cryptotls.TLS_AES_128_GCM_SHA256)
suite.writeBody(body) suite.writeBody(body)
@@ -263,7 +263,6 @@ func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadCompressionMe
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipCompressionMethods() { func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipCompressionMethods() {
body := make([]byte, 2+fake.RandomLen+1+2+2+1) body := make([]byte, 2+fake.RandomLen+1+2+2+1)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2) binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2:], cryptotls.TLS_AES_128_GCM_SHA256)
body[2+fake.RandomLen+1+2+2] = 1 body[2+fake.RandomLen+1+2+2] = 1
suite.writeBody(body) suite.writeBody(body)
@@ -299,7 +298,6 @@ func (suite *ParseClientHelloSNITestSuite) writeExtensions(extensions []byte) {
// cipherSuite(2) + compressionLen(1) + compression(1) = 41 // cipherSuite(2) + compressionLen(1) + compression(1) = 41
body := make([]byte, 41) body := make([]byte, 41)
binary.BigEndian.PutUint16(body[35:], 2) binary.BigEndian.PutUint16(body[35:], 2)
binary.BigEndian.PutUint16(body[37:], cryptotls.TLS_AES_128_GCM_SHA256)
body[39] = 1 body[39] = 1
suite.readBuf.Write(body) suite.readBuf.Write(body)
@@ -395,268 +393,3 @@ func TestParseClientHelloSNI(t *testing.T) {
t.Parallel() t.Parallel()
suite.Run(t, &ParseClientHelloSNITestSuite{}) suite.Run(t, &ParseClientHelloSNITestSuite{})
} }
// fragmentTLSRecord splits a single TLS record into n TLS records by
// dividing the payload into roughly equal parts. Each part gets its own
// TLS record header with the same record type and version.
func fragmentTLSRecord(t testing.TB, full []byte, n int) []byte {
t.Helper()
recordType := full[0]
version := full[1:3]
payload := full[tls.SizeHeader:]
chunkSize := len(payload) / n
result := &bytes.Buffer{}
for i := 0; i < n; i++ {
start := i * chunkSize
end := start + chunkSize
if i == n-1 {
end = len(payload)
}
chunk := payload[start:end]
result.WriteByte(recordType)
result.Write(version)
require.NoError(t, binary.Write(result, binary.BigEndian, uint16(len(chunk))))
result.Write(chunk)
}
return result.Bytes()
}
// splitPayloadAt creates two TLS records from a single record by splitting
// the payload at the given byte position.
func splitPayloadAt(t testing.TB, full []byte, pos int) []byte {
t.Helper()
payload := full[tls.SizeHeader:]
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(t, binary.Write(buf, binary.BigEndian, uint16(pos)))
buf.Write(payload[:pos])
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(t, binary.Write(buf, binary.BigEndian, uint16(len(payload)-pos)))
buf.Write(payload[pos:])
return buf.Bytes()
}
type ParseClientHelloFragmentedTestSuite struct {
suite.Suite
secret mtglib.Secret
snapshot *clientHelloSnapshot
}
func (s *ParseClientHelloFragmentedTestSuite) SetupSuite() {
parsed, err := mtglib.ParseSecret(
"ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d",
)
require.NoError(s.T(), err)
s.secret = parsed
fileData, err := os.ReadFile("testdata/client-hello-ok-19dfe38384b9884b.json")
require.NoError(s.T(), err)
s.snapshot = &clientHelloSnapshot{}
require.NoError(s.T(), json.Unmarshal(fileData, s.snapshot))
}
func (s *ParseClientHelloFragmentedTestSuite) makeConn(data []byte) *parseClientHelloConnMock {
readBuf := &bytes.Buffer{}
readBuf.Write(data)
connMock := &parseClientHelloConnMock{
readBuf: readBuf,
}
return connMock
}
func (s *ParseClientHelloFragmentedTestSuite) TestReassemblySuccess() {
full := s.snapshot.GetFull()
tests := []struct {
name string
data []byte
}{
{"two equal fragments", fragmentTLSRecord(s.T(), full, 2)},
{"three equal fragments", fragmentTLSRecord(s.T(), full, 3)},
{"single byte first fragment", splitPayloadAt(s.T(), full, 1)},
{"three byte first fragment", splitPayloadAt(s.T(), full, 3)},
}
for _, tt := range tests {
s.Run(tt.name, func() {
connMock := s.makeConn(tt.data)
defer connMock.AssertExpectations(s.T())
hello, err := fake.ReadClientHello(
connMock,
s.secret.Key[:],
s.secret.Host,
TolerateTime,
)
s.Require().NoError(err)
s.Equal(s.snapshot.GetRandom(), hello.Random[:])
s.Equal(s.snapshot.GetSessionID(), hello.SessionID)
s.Equal(uint16(s.snapshot.CipherSuite), hello.CipherSuite)
})
}
}
func (s *ParseClientHelloFragmentedTestSuite) TestReassemblyErrors() {
full := s.snapshot.GetFull()
payload := full[tls.SizeHeader:]
tests := []struct {
name string
buildData func() []byte
errMsg string
}{
{
name: "wrong continuation record type",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Wrong type: application data instead of handshake
buf.WriteByte(tls.TypeApplicationData)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(payload)-10)))
buf.Write(payload[10:])
return buf.Bytes()
},
errMsg: "unexpected record type",
},
{
name: "too many continuation records",
buildData: func() []byte {
// Handshake header claiming 256 bytes, but we only send 1 byte per continuation
handshakePayload := []byte{0x01, 0x00, 0x01, 0x00}
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 1})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(handshakePayload))))
buf.Write(handshakePayload)
for range 11 {
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 1})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(1)))
buf.WriteByte(0xAB)
}
return buf.Bytes()
},
errMsg: "too many fragments",
},
{
name: "zero-length continuation record",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Valid header but zero-length payload
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(0)))
return buf.Bytes()
},
errMsg: "cannot read record header",
},
{
name: "wrong continuation record version",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Wrong version: 3.3 instead of 3.1
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 3})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(payload)-10)))
buf.Write(payload[10:])
return buf.Bytes()
},
errMsg: "unexpected protocol version",
},
{
name: "handshake message too large",
buildData: func() []byte {
// Handshake header claiming 0x010000 (65536) bytes — exceeds 0xFFFF limit
handshakePayload := []byte{0x01, 0x01, 0x00, 0x00}
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 1})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(handshakePayload))))
buf.Write(handshakePayload)
return buf.Bytes()
},
errMsg: "cannot read record header",
},
{
name: "truncated continuation record header",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Connection ends mid-header (only 2 bytes)
buf.WriteByte(tls.TypeHandshake)
buf.WriteByte(3)
return buf.Bytes()
},
errMsg: "cannot read record header",
},
{
name: "truncated continuation record payload",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Claims 100 bytes but no payload follows
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(100)))
return buf.Bytes()
},
errMsg: "EOF",
},
}
for _, tt := range tests {
s.Run(tt.name, func() {
connMock := s.makeConn(tt.buildData())
defer connMock.AssertExpectations(s.T())
_, err := fake.ReadClientHello(
connMock,
s.secret.Key[:],
s.secret.Host,
TolerateTime,
)
s.ErrorContains(err, tt.errMsg)
})
}
}
func TestParseClientHelloFragmented(t *testing.T) {
t.Parallel()
suite.Run(t, &ParseClientHelloFragmentedTestSuite{})
}
+10 -1
View File
@@ -2,6 +2,15 @@ package fake
import ( import (
"errors" "errors"
"time"
) )
var ErrBadDigest = errors.New("incorrect client random") const (
ClientHelloReadTimeout = 5 * time.Second
)
var (
resetDeadline time.Time
ErrBadDigest = errors.New("incorrect client random")
)
+1 -1
View File
@@ -58,7 +58,7 @@ func (suite *SendServerHelloTestSuite) TestRecordStructure() {
recordType, length, err := tls.ReadRecord(suite.buf, &rec) recordType, length, err := tls.ReadRecord(suite.buf, &rec)
suite.NoError(err) suite.NoError(err)
suite.Equal(byte(tls.TypeApplicationData), recordType) suite.Equal(byte(tls.TypeApplicationData), recordType)
suite.GreaterOrEqual(length, int64(2500)) suite.Greater(length, int64(2500))
suite.Empty(suite.buf.Bytes()) suite.Empty(suite.buf.Bytes())
} }
@@ -1,8 +0,0 @@
{
"time": 1617181365,
"random": "w4TaDfYg/aUKdx1oi68vxMKvHJczRNvtRRppLETzeNE=",
"sessionId": "St2BZ2uHMFn3B2trD1jfdtpjoJOOg6JBeLhFcyCMCq4=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "FgMBAgIBAAH+AwPDhNoN9iD9pQp3HWiLry/Ewq8clzNE2+1FGmksRPN40SBK3YFna4cwWfcHa2sPWN922mOgk46DokF4uEVzIIwKrgA2WloTAxMBEwLALMArwCTAI8AKwAnMqcAwwC/AKMAnwBTAE8yoAJ0AnAA9ADwANQAvwAjAEgAKAQABf/8BAAEAAAAAGwAZAAAWc3RvcmFnZS5nb29nbGVhcGlzLmNvbQAXAAAADQAYABYEAwgEBAEFAwIDCAUIBQUBCAYGAQIBAAUABQEAAAAAM3QAAAASAAAAEAAwAC4CaDIFaDItMTYFaDItMTUFaDItMTQIc3BkeS8zLjEGc3BkeS8zCGh0dHAvMS4xAAsAAgEAADMAJgAkAB0AIAf+6C8fSRJSAC7CyUvdR9kDclNR9KLCsCFHpVZ3bC8iAC0AAgEBACsACQgDBAMDAwIDAQAKAAoACAAdABcAGAAZABUAoQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
}
-158
View File
@@ -1,158 +0,0 @@
package fake
import (
"bytes"
"encoding/binary"
"errors"
"fmt"
"io"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
const (
maxFragmentsCount = 10
)
var ErrTooManyFragments = errors.New("too many fragments")
// https://datatracker.ietf.org/doc/html/rfc5246#section-6.2.1
// client hello can be fragmented in a series of packets:
//
// Bytes on the wire:
//
// 16 03 01 00 F8 01 00 00 F4 03 03 [32 bytes random] [session_id] [ciphers] [SNI...]
// ├─────────────┤├──────────────────────────────────────────────────────────────────┤
//
// TLS record Payload (248 bytes)
// header (5B)
//
// 16 = Handshake
// 03 01 = TLS 1.0 (record layer version)
// 00 F8 = 248 bytes follow
//
// 01 = ClientHello (handshake type)
// 00 00 F4 = 244 bytes of handshake body
// 03 03 = TLS 1.2 (actual protocol version)
// ...rest of ClientHello...
//
// Fragmented record look like:
//
// Record 1:
//
// 16 03 01 00 03 01 00 00
// ├─────────────┤├──────┤
//
// TLS header 3 bytes of payload
//
// 16 = Handshake
// 03 01 = TLS 1.0
// 00 03 = only 3 bytes follow
//
// 01 = ClientHello type
// 00 00 = first 2 bytes of the uint24 length (INCOMPLETE!)
//
// Record 2:
// 16 03 01 00 F5 F4 03 03 [32 bytes random] [session_id] [ciphers] [SNI...]
// ├─────────────┤├────────────────────────────────────────────────────────────┤
//
// TLS header remaining 245 bytes of payload
//
// 16 = Handshake
// 03 01 = TLS 1.0
// 00 F5 = 245 bytes follow
//
// F4 = last byte of uint24 length (now complete: 00 00 F4 = 244)
// 03 03 = TLS 1.2
// ...rest of ClientHello continues...
//
// So it means that there could be a series of handshake packets of different
// lengths. The goal of this function is to concatenate these fragments.
type fragmentedHandshakeReader struct {
r io.Reader
buf bytes.Buffer
readFragments int
}
func (f *fragmentedHandshakeReader) Read(p []byte) (int, error) {
if n, err := f.buf.Read(p); err == nil {
return n, nil
}
f.buf.Reset()
for f.buf.Len() == 0 {
if f.readFragments > maxFragmentsCount {
return 0, ErrTooManyFragments
}
if err := f.parseNextFragment(); err != nil {
return 0, err
}
f.readFragments++
}
return f.buf.Read(p)
}
func (f *fragmentedHandshakeReader) parseNextFragment() error {
// record_type(1) + version(2) + size(2)
// 16 - type is 0x16 (handshake record)
// 03 01 - protocol version is "3,1" (also known as TLS 1.0)
// 00 f8 - 0xF8 (248) bytes of handshake message follows
header := [1 + 2 + 2]byte{}
if _, err := io.ReadFull(f.r, header[:]); err != nil {
return fmt.Errorf("cannot read record header: %w", err)
}
if header[0] != tls.TypeHandshake {
return fmt.Errorf("unexpected record type %#x", header[0])
}
if header[1] != 3 || header[2] != 1 {
return fmt.Errorf("unexpected protocol version %#x %#x", header[1], header[2])
}
length := int64(binary.BigEndian.Uint16(header[3:]))
_, err := io.CopyN(&f.buf, f.r, length)
return err
}
func parseClientHello(r io.Reader) (*bytes.Buffer, *bytes.Buffer, error) {
r = &fragmentedHandshakeReader{r: r}
header := [1 + 3]byte{}
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, nil, fmt.Errorf("cannot read handshake header: %w", err)
}
if header[0] != TypeHandshakeClient {
return nil, nil, fmt.Errorf("incorrect handshake type: %#x", header[0])
}
// unfortunately there is not uint24 in golang, so we just reuse header
header[0] = 0
length := int64(binary.BigEndian.Uint32(header[:]))
clientHelloCopy := &bytes.Buffer{}
clientHelloCopy.Write([]byte{tls.TypeHandshake, 3, 1})
binary.Write( //nolint: errcheck
clientHelloCopy,
binary.BigEndian,
// 1 for handshake type
// 3 for handshake length
uint16(1+3+length),
)
clientHelloCopy.WriteByte(TypeHandshakeClient)
clientHelloCopy.Write(header[1:])
handshakeCopy := &bytes.Buffer{}
writer := io.MultiWriter(clientHelloCopy, handshakeCopy)
_, err := io.CopyN(writer, r, length)
return clientHelloCopy, handshakeCopy, err
}
+4 -20
View File
@@ -28,7 +28,6 @@ type Proxy struct {
allowFallbackOnUnknownDC bool allowFallbackOnUnknownDC bool
tolerateTimeSkewness time.Duration tolerateTimeSkewness time.Duration
idleTimeout time.Duration idleTimeout time.Duration
handshakeTimeout time.Duration
domainFrontingPort int domainFrontingPort int
domainFrontingIP string domainFrontingIP string
domainFrontingProxyProtocol bool domainFrontingProxyProtocol bool
@@ -67,11 +66,6 @@ func (p *Proxy) ServeConn(conn essentials.Conn) {
ctx := newStreamContext(p.ctx, p.logger, conn) ctx := newStreamContext(p.ctx, p.logger, conn)
defer ctx.Close() defer ctx.Close()
if err := ctx.clientConn.SetDeadline(time.Now().Add(p.handshakeTimeout)); err != nil {
ctx.logger.WarningError("cannot set handshake timeout", err)
return
}
stop := context.AfterFunc(ctx, func() { stop := context.AfterFunc(ctx, func() {
ctx.Close() ctx.Close()
}) })
@@ -103,23 +97,16 @@ func (p *Proxy) ServeConn(conn essentials.Conn) {
return return
} }
if err := ctx.clientConn.SetDeadline(time.Time{}); err != nil {
ctx.logger.WarningError("cannot set deadline", err)
return
}
if err := p.doTelegramCall(ctx); err != nil { if err := p.doTelegramCall(ctx); err != nil {
ctx.logger.WarningError("cannot dial to telegram", err) ctx.logger.WarningError("cannot dial to telegram", err)
return return
} }
tracker := newIdleTracker(p.idleTimeout)
relay.Relay( relay.Relay(
ctx, ctx,
ctx.logger.Named("relay"), ctx.logger.Named("relay"),
connIdleTimeout{Conn: ctx.telegramConn, tracker: tracker}, connIdleTimeout{Conn: ctx.telegramConn, timeout: p.idleTimeout},
connIdleTimeout{Conn: ctx.clientConn, tracker: tracker}, connIdleTimeout{Conn: ctx.clientConn, timeout: p.idleTimeout},
) )
} }
@@ -318,13 +305,11 @@ func (p *Proxy) doDomainFronting(ctx *streamContext, conn *connRewind) {
stream: p.eventStream, stream: p.eventStream,
} }
tracker := newIdleTracker(p.idleTimeout)
relay.Relay( relay.Relay(
ctx, ctx,
ctx.logger.Named("domain-fronting"), ctx.logger.Named("domain-fronting"),
connIdleTimeout{Conn: frontConn, tracker: tracker}, connIdleTimeout{Conn: frontConn, timeout: p.idleTimeout},
connIdleTimeout{Conn: conn, tracker: tracker}, connIdleTimeout{Conn: conn, timeout: p.idleTimeout},
) )
} }
@@ -357,7 +342,6 @@ func NewProxy(opts ProxyOpts) (*Proxy, error) {
domainFrontingIP: opts.DomainFrontingIP, domainFrontingIP: opts.DomainFrontingIP,
tolerateTimeSkewness: opts.getTolerateTimeSkewness(), tolerateTimeSkewness: opts.getTolerateTimeSkewness(),
idleTimeout: opts.getIdleTimeout(), idleTimeout: opts.getIdleTimeout(),
handshakeTimeout: opts.getHandshakeTimeout(),
allowFallbackOnUnknownDC: opts.AllowFallbackOnUnknownDC, allowFallbackOnUnknownDC: opts.AllowFallbackOnUnknownDC,
telegram: tg, telegram: tg,
doppelGanger: doppel.NewGanger( doppelGanger: doppel.NewGanger(
+2 -15
View File
@@ -70,12 +70,6 @@ type ProxyOpts struct {
// This is an optional setting. // This is an optional setting.
IdleTimeout time.Duration IdleTimeout time.Duration
// HandshakeTimeout is a timeout during which all handshake ceremonies must
// be completed, otherwise this process will be aborted
//
// This is an optional setting.
HandshakeTimeout time.Duration
// TolerateTimeSkewness is a time boundary that defines a time range where // TolerateTimeSkewness is a time boundary that defines a time range where
// faketls timestamp is acceptable. // faketls timestamp is acceptable.
// //
@@ -166,6 +160,7 @@ type ProxyOpts struct {
// DoppelGangerDRS defines if TLS Dynamic Record Sizing is active. // DoppelGangerDRS defines if TLS Dynamic Record Sizing is active.
DoppelGangerDRS bool DoppelGangerDRS bool
} }
func (p ProxyOpts) valid() error { func (p ProxyOpts) valid() error {
@@ -221,17 +216,9 @@ func (p ProxyOpts) getPreferIP() string {
return p.PreferIP return p.PreferIP
} }
func (p ProxyOpts) getHandshakeTimeout() time.Duration {
if p.HandshakeTimeout == 0 {
return DefaultHandshakeTimeout
}
return p.HandshakeTimeout
}
func (p ProxyOpts) getIdleTimeout() time.Duration { func (p ProxyOpts) getIdleTimeout() time.Duration {
if p.IdleTimeout == 0 { if p.IdleTimeout == 0 {
return DefaultIdleTimeout return time.Minute
} }
return p.IdleTimeout return p.IdleTimeout
+1 -1
View File
@@ -175,7 +175,7 @@ func (suite *ProxyTestSuite) TestHTTPSRequest() {
addr := fmt.Sprintf("https://%s/headers", suite.ProxyAddress()) addr := fmt.Sprintf("https://%s/headers", suite.ProxyAddress())
resp, err := client.Get(addr) //nolint: noctx resp, err := client.Get(addr) //nolint: noctx
suite.Require().NoError(err) suite.NoError(err)
defer resp.Body.Close() //nolint: errcheck defer resp.Body.Close() //nolint: errcheck
-14
View File
@@ -36,22 +36,8 @@ const (
// DefaultTCPKeepAlivePeriod defines a time period between 2 consequitive // DefaultTCPKeepAlivePeriod defines a time period between 2 consequitive
// probes. // probes.
//
// Deprecated: use DefaultKeepAliveIdle and DefaultKeepAliveInterval instead.
DefaultTCPKeepAlivePeriod = 10 * time.Second DefaultTCPKeepAlivePeriod = 10 * time.Second
// DefaultKeepAliveIdle is the time a connection must be idle before
// the first keepalive probe is sent.
DefaultKeepAliveIdle = 30 * time.Second
// DefaultKeepAliveInterval is the time between consecutive keepalive
// probes.
DefaultKeepAliveInterval = 10 * time.Second
// DefaultKeepAliveCount is the number of unacknowledged probes before
// the connection is considered dead.
DefaultKeepAliveCount = 3
// ProxyDialerOpenThreshold is used for load balancing SOCKS5 dialer only. // ProxyDialerOpenThreshold is used for load balancing SOCKS5 dialer only.
// //
// This dialer uses circuit breaker with of 3 stages: OPEN, HALF_OPEN and // This dialer uses circuit breaker with of 3 stages: OPEN, HALF_OPEN and
+2 -7
View File
@@ -20,13 +20,8 @@ func SetServerSocketOptions(conn net.Conn, bufferSize int) error {
} }
func setCommonSocketOptions(conn *net.TCPConn) error { func setCommonSocketOptions(conn *net.TCPConn) error {
if err := applyKeepAlive(conn, net.KeepAliveConfig{ if err := conn.SetKeepAlivePeriod(DefaultTCPKeepAlivePeriod); err != nil {
Enable: true, return fmt.Errorf("cannot set time period of TCP keepalive probes: %w", err)
Idle: DefaultKeepAliveIdle,
Interval: DefaultKeepAliveInterval,
Count: DefaultKeepAliveCount,
}); err != nil {
return fmt.Errorf("cannot configure TCP keepalive: %w", err)
} }
if err := conn.SetLinger(tcpLingerTimeout); err != nil { if err := conn.SetLinger(tcpLingerTimeout); err != nil {
-11
View File
@@ -1,11 +0,0 @@
//go:build !openbsd
package network
import "net"
// applyKeepAlive enables TCP keepalive on conn and applies the per-socket
// idle/interval/count tuning from cfg.
func applyKeepAlive(conn *net.TCPConn, cfg net.KeepAliveConfig) error {
return conn.SetKeepAliveConfig(cfg) //nolint: wrapcheck
}
-20
View File
@@ -1,20 +0,0 @@
package network
import "net"
// applyKeepAlive enables (or disables) TCP keepalive on conn.
//
// OpenBSD has no user-settable per-socket TCP keepalive options: TCP_KEEPIDLE,
// TCP_KEEPINTVL and TCP_KEEPCNT do not exist on OpenBSD, and Go's
// (*TCPConn).SetKeepAliveConfig therefore returns ENOPROTOOPT ("protocol not
// available") for any non-negative Idle/Interval/Count value (see
// src/net/tcpsockopt_openbsd.go in the Go source tree). Calling
// SetKeepAliveConfig with mtg's defaults (zero values) breaks every accepted
// listener connection and every outbound dial on OpenBSD.
//
// On OpenBSD we only flip SO_KEEPALIVE on or off; the keepalive timing is
// controlled system-wide via the sysctl knobs net.inet.tcp.keepidle and
// net.inet.tcp.keepintvl.
func applyKeepAlive(conn *net.TCPConn, cfg net.KeepAliveConfig) error {
return conn.SetKeepAlive(cfg.Enable) //nolint: wrapcheck
}
-93
View File
@@ -1,93 +0,0 @@
//go:build linux || darwin
// +build linux darwin
package network_test
import (
"net"
"runtime"
"syscall"
"testing"
"time"
"github.com/9seconds/mtg/v2/network"
"github.com/stretchr/testify/require"
"golang.org/x/sys/unix"
)
func tcpKeepIdleOption() int {
if runtime.GOOS == "darwin" {
return 0x10 // TCP_KEEPALIVE on macOS
}
return 0x4 // TCP_KEEPIDLE on Linux
}
func TestSetClientSocketOptionsKeepAlive(t *testing.T) {
t.Parallel()
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() {
err := listener.Close()
require.NoError(t, err)
}()
type dialResult struct {
conn net.Conn
err error
}
dialDone := make(chan dialResult, 1)
go func() {
c, err := net.Dial("tcp", listener.Addr().String())
dialDone <- dialResult{conn: c, err: err}
}()
tcpListener, ok := listener.(*net.TCPListener)
require.True(t, ok, "listener must be a *net.TCPListener")
require.NoError(t, tcpListener.SetDeadline(time.Now().Add(5*time.Second)))
accepted, err := listener.Accept()
require.NoError(t, err)
defer func() {
err := accepted.Close()
require.NoError(t, err)
}()
dr := <-dialDone
require.NoError(t, dr.err)
defer func() {
err := dr.conn.Close()
require.NoError(t, err)
}()
err = network.SetClientSocketOptions(accepted, 0)
require.NoError(t, err)
tcpConn := accepted.(*net.TCPConn)
rawConn, err := tcpConn.SyscallConn()
require.NoError(t, err)
err = rawConn.Control(func(fd uintptr) {
val, err := unix.GetsockoptInt(int(fd), syscall.SOL_SOCKET, syscall.SO_KEEPALIVE)
require.NoError(t, err)
require.NotEqual(t, 0, val, "SO_KEEPALIVE should be enabled")
idle, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, tcpKeepIdleOption())
require.NoError(t, err)
require.Equal(t, int(network.DefaultKeepAliveIdle.Seconds()), idle)
interval, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, unix.TCP_KEEPINTVL)
require.NoError(t, err)
require.Equal(t, int(network.DefaultKeepAliveInterval.Seconds()), interval)
count, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, unix.TCP_KEEPCNT)
require.NoError(t, err)
require.Equal(t, network.DefaultKeepAliveCount, count)
})
require.NoError(t, err)
}
+1 -1
View File
@@ -25,7 +25,7 @@ func (suite *BaseHTTPTestSuite) SetupSuite() {
} }
func (suite *BaseHTTPTestSuite) SetupTest() { func (suite *BaseHTTPTestSuite) SetupTest() {
suite.client = network.New(nil, "mtg/1", 0, 0, 0, network.DefaultKeepAliveConfig).MakeHTTPClient(nil) suite.client = network.New(nil, "mtg/1", 0, 0, 0).MakeHTTPClient(nil)
} }
func (suite *BaseHTTPTestSuite) TestGet() { func (suite *BaseHTTPTestSuite) TestGet() {
+1 -1
View File
@@ -19,7 +19,7 @@ type BaseNetworkTestSuite struct {
func (suite *BaseNetworkTestSuite) SetupSuite() { func (suite *BaseNetworkTestSuite) SetupSuite() {
suite.EchoServerTestSuite.SetupSuite() suite.EchoServerTestSuite.SetupSuite()
suite.net = network.New(nil, "agent", 0, 0, 0, network.DefaultKeepAliveConfig) suite.net = network.New(nil, "agent", 0, 0, 0)
} }
func (suite *BaseNetworkTestSuite) TestDialUnknownNetwork() { func (suite *BaseNetworkTestSuite) TestDialUnknownNetwork() {
+1 -43
View File
@@ -11,7 +11,6 @@ package network
import ( import (
"errors" "errors"
"net"
"time" "time"
) )
@@ -27,55 +26,14 @@ const (
// DefaultTCPKeepAlivePeriod defines a time period between 2 consecuitive // DefaultTCPKeepAlivePeriod defines a time period between 2 consecuitive
// probes. // probes.
//
// Deprecated: use DefaultKeepAliveConfig
DefaultTCPKeepAlivePeriod = 10 * time.Second DefaultTCPKeepAlivePeriod = 10 * time.Second
// DefaultKeepAliveIdle is the time a connection must be idle before
// the first keepalive probe is sent.
//
// Deprecated: use DefaultKeepAliveConfig
DefaultKeepAliveIdle = 30 * time.Second
// DefaultKeepAliveInterval is the time between consecutive keepalive
// probes.
//
// Deprecated: use DefaultKeepAliveConfig
DefaultKeepAliveInterval = 10 * time.Second
// DefaultKeepAliveCount is the number of unacknowledged probes before
// the connection is considered dead.
//
// Deprecated: use DefaultKeepAliveConfig
DefaultKeepAliveCount = 3
// User Agent to use in HTTP client. // User Agent to use in HTTP client.
UserAgent = "curl/8.5.0" UserAgent = "curl/8.5.0"
// tcpLingerTimeout defines a number of seconds to wait for sending // tcpLingerTimeout defines a number of seconds to wait for sending
// unacknowledged data. // unacknowledged data.
tcpLingerTimeout = 1 tcpLingerTimeout = 1
// tcpNotSentLowat limits the amount of unsent data queued in the
// kernel write buffer per socket. When the unsent data drops below
// this threshold, the socket becomes writable again. This reduces
// per-connection memory usage and bufferbloat by applying
// back-pressure to the relay loop instead of piling up data in
// kernel buffers.
tcpNotSentLowat = 128 * 1024
) )
var ( var ErrCannotDial = errors.New("cannot dial to any address")
ErrCannotDial = errors.New("cannot dial to any address")
// DefaultKeepAliveConfig defines a default configuration for
// keep alive settings. As per official documentation, if keep alive
// is enabled, then:
//
// Idle = 15 * time.Second
// Interval = 15 * time.Second
// Count = 9
DefaultKeepAliveConfig = net.KeepAliveConfig{
Enable: true,
}
)
+7 -10
View File
@@ -14,10 +14,9 @@ import (
type network struct { type network struct {
net.Dialer net.Dialer
keepAliveConfig net.KeepAliveConfig httpTimeout time.Duration
httpTimeout time.Duration idleTimeout time.Duration
idleTimeout time.Duration userAgent string
userAgent string
} }
func (n *network) Dial(network, address string) (essentials.Conn, error) { func (n *network) Dial(network, address string) (essentials.Conn, error) {
@@ -38,7 +37,7 @@ func (n *network) DialContext(ctx context.Context, network, address string) (ess
tcpConn := conn.(*net.TCPConn) tcpConn := conn.(*net.TCPConn)
return tcpConn, setCommonSocketOptions(tcpConn, n.keepAliveConfig) return tcpConn, setCommonSocketOptions(tcpConn)
} }
func (n *network) MakeHTTPClient( func (n *network) MakeHTTPClient(
@@ -72,7 +71,6 @@ func New(
tcpTimeout, tcpTimeout,
httpTimeout, httpTimeout,
idleTimeout time.Duration, idleTimeout time.Duration,
keepAliveConfig net.KeepAliveConfig,
) mtglib.Network { ) mtglib.Network {
if dnsResolver == nil { if dnsResolver == nil {
dnsResolver = net.DefaultResolver dnsResolver = net.DefaultResolver
@@ -88,9 +86,8 @@ func New(
Resolver: dnsResolver, Resolver: dnsResolver,
FallbackDelay: -1, FallbackDelay: -1,
}, },
userAgent: userAgent, userAgent: userAgent,
idleTimeout: idleTimeout, idleTimeout: idleTimeout,
httpTimeout: httpTimeout, httpTimeout: httpTimeout,
keepAliveConfig: keepAliveConfig,
} }
} }
+3 -7
View File
@@ -5,9 +5,9 @@ import (
"net" "net"
) )
func setCommonSocketOptions(conn *net.TCPConn, keepAliveConfig net.KeepAliveConfig) error { func setCommonSocketOptions(conn *net.TCPConn) error {
if err := applyKeepAlive(conn, keepAliveConfig); err != nil { if err := conn.SetKeepAlivePeriod(DefaultTCPKeepAlivePeriod); err != nil {
return fmt.Errorf("cannot configure TCP keepalive: %w", err) return fmt.Errorf("cannot set time period of TCP keepalive probes: %w", err)
} }
if err := conn.SetLinger(tcpLingerTimeout); err != nil { if err := conn.SetLinger(tcpLingerTimeout); err != nil {
@@ -23,9 +23,5 @@ func setCommonSocketOptions(conn *net.TCPConn, keepAliveConfig net.KeepAliveConf
return fmt.Errorf("cannot setup SO_REUSEADDR/PORT: %w", err) return fmt.Errorf("cannot setup SO_REUSEADDR/PORT: %w", err)
} }
setCongestionControl(rawConn)
setTCPUserTimeout(rawConn, keepAliveConfig)
setNotSentLowat(rawConn)
return nil return nil
} }
-20
View File
@@ -1,20 +0,0 @@
//go:build linux
package network
import (
"syscall"
"golang.org/x/sys/unix"
)
// setCongestionControl sets BBR as the TCP congestion control algorithm.
// BBR provides better throughput over lossy and high-latency links compared
// to the default cubic, which is especially beneficial for mobile and
// home internet clients. This is best-effort: silently ignored if the
// kernel does not have tcp_bbr available.
func setCongestionControl(conn syscall.RawConn) {
conn.Control(func(fd uintptr) { //nolint: errcheck
unix.SetsockoptString(int(fd), unix.IPPROTO_TCP, unix.TCP_CONGESTION, "bbr") //nolint: errcheck
})
}
-7
View File
@@ -1,7 +0,0 @@
//go:build !linux
package network
import "syscall"
func setCongestionControl(conn syscall.RawConn) {}
-11
View File
@@ -1,11 +0,0 @@
//go:build !openbsd
package network
import "net"
// applyKeepAlive enables TCP keepalive on conn and applies the per-socket
// idle/interval/count tuning from cfg.
func applyKeepAlive(conn *net.TCPConn, cfg net.KeepAliveConfig) error {
return conn.SetKeepAliveConfig(cfg) //nolint: wrapcheck
}
-20
View File
@@ -1,20 +0,0 @@
package network
import "net"
// applyKeepAlive enables (or disables) TCP keepalive on conn.
//
// OpenBSD has no user-settable per-socket TCP keepalive options: TCP_KEEPIDLE,
// TCP_KEEPINTVL and TCP_KEEPCNT do not exist on OpenBSD, and Go's
// (*TCPConn).SetKeepAliveConfig therefore returns ENOPROTOOPT ("protocol not
// available") for any non-negative Idle/Interval/Count value (see
// src/net/tcpsockopt_openbsd.go in the Go source tree). Calling
// SetKeepAliveConfig with mtg's defaults (zero values) breaks every accepted
// listener connection and every outbound dial on OpenBSD.
//
// On OpenBSD we only flip SO_KEEPALIVE on or off; the keepalive timing is
// controlled system-wide via the sysctl knobs net.inet.tcp.keepidle and
// net.inet.tcp.keepintvl.
func applyKeepAlive(conn *net.TCPConn, cfg net.KeepAliveConfig) error {
return conn.SetKeepAlive(cfg.Enable) //nolint: wrapcheck
}
-20
View File
@@ -1,20 +0,0 @@
//go:build linux || darwin
package network
import (
"syscall"
"golang.org/x/sys/unix"
)
// setNotSentLowat sets TCP_NOTSENT_LOWAT which limits the amount of
// unsent data queued in the kernel write buffer. Once unsent data drops
// below this threshold the socket becomes writable again, applying
// back-pressure to the relay loop instead of piling up data in kernel
// buffers. This reduces per-connection memory and bufferbloat.
func setNotSentLowat(conn syscall.RawConn) {
conn.Control(func(fd uintptr) { //nolint: errcheck
unix.SetsockoptInt(int(fd), unix.IPPROTO_TCP, unix.TCP_NOTSENT_LOWAT, tcpNotSentLowat) //nolint: errcheck
})
}
-7
View File
@@ -1,7 +0,0 @@
//go:build !linux && !darwin
package network
import "syscall"
func setNotSentLowat(conn syscall.RawConn) {}
-92
View File
@@ -1,92 +0,0 @@
//go:build linux || darwin
// +build linux darwin
package network
import (
"net"
"runtime"
"syscall"
"testing"
"time"
"github.com/stretchr/testify/require"
"golang.org/x/sys/unix"
)
func tcpKeepIdleOption() int {
if runtime.GOOS == "darwin" {
return 0x10 // TCP_KEEPALIVE on macOS
}
return 0x4 // TCP_KEEPIDLE on Linux
}
func TestSetCommonSocketOptionsKeepAlive(t *testing.T) {
t.Parallel()
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() {
err := listener.Close()
require.NoError(t, err)
}()
type dialResult struct {
conn net.Conn
err error
}
dialDone := make(chan dialResult, 1)
go func() {
c, err := net.Dial("tcp", listener.Addr().String())
dialDone <- dialResult{conn: c, err: err}
}()
tcpListener, ok := listener.(*net.TCPListener)
require.True(t, ok, "listener must be a *net.TCPListener")
require.NoError(t, tcpListener.SetDeadline(time.Now().Add(5*time.Second)))
accepted, err := listener.Accept()
require.NoError(t, err)
defer func() {
err := accepted.Close()
require.NoError(t, err)
}()
dr := <-dialDone
require.NoError(t, dr.err)
defer func() {
err := dr.conn.Close()
require.NoError(t, err)
}()
tcpConn := accepted.(*net.TCPConn)
err = setCommonSocketOptions(tcpConn, DefaultKeepAliveConfig)
require.NoError(t, err)
rawConn, err := tcpConn.SyscallConn()
require.NoError(t, err)
err = rawConn.Control(func(fd uintptr) {
val, err := unix.GetsockoptInt(int(fd), syscall.SOL_SOCKET, syscall.SO_KEEPALIVE)
require.NoError(t, err)
require.NotEqual(t, 0, val, "SO_KEEPALIVE should be enabled")
idle, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, tcpKeepIdleOption())
require.NoError(t, err)
require.Equal(t, 15, idle, "keepalive idle should match DefaultKeepAliveIdle")
interval, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, unix.TCP_KEEPINTVL)
require.NoError(t, err)
require.Equal(t, 15, interval, "keepalive interval should match DefaultKeepAliveInterval")
count, err := unix.GetsockoptInt(int(fd), syscall.IPPROTO_TCP, unix.TCP_KEEPCNT)
require.NoError(t, err)
require.Equal(t, 9, count, "keepalive count should match DefaultKeepAliveCount")
})
require.NoError(t, err)
}
@@ -1,4 +1,5 @@
//go:build !windows //go:build !windows
// +build !windows
package network package network
-48
View File
@@ -1,48 +0,0 @@
//go:build linux
package network
import (
"net"
"syscall"
"time"
"golang.org/x/sys/unix"
)
// Go runtime defaults for KeepAliveConfig when fields are zero.
const (
goDefaultKeepAliveIdle = 15 * time.Second
goDefaultKeepAliveInterval = 15 * time.Second
goDefaultKeepAliveCount = 9
)
// setTCPUserTimeout sets TCP_USER_TIMEOUT on a socket. If transmitted
// data remains unacknowledged for this long, the kernel closes the
// connection. As recommended by Cloudflare
// (https://blog.cloudflare.com/when-tcp-sockets-refuse-to-die/),
// the value is computed as: keepidle + keepintvl * keepcnt. This
// ensures TCP_USER_TIMEOUT and keepalives agree on when to give up.
// Best-effort: silently ignored if unsupported.
func setTCPUserTimeout(conn syscall.RawConn, cfg net.KeepAliveConfig) {
idle := cfg.Idle
if idle == 0 {
idle = goDefaultKeepAliveIdle
}
interval := cfg.Interval
if interval == 0 {
interval = goDefaultKeepAliveInterval
}
count := cfg.Count
if count == 0 {
count = goDefaultKeepAliveCount
}
timeout := idle + interval*time.Duration(count)
conn.Control(func(fd uintptr) { //nolint: errcheck
unix.SetsockoptInt(int(fd), unix.IPPROTO_TCP, unix.TCP_USER_TIMEOUT, int(timeout.Milliseconds())) //nolint: errcheck
})
}
-10
View File
@@ -1,10 +0,0 @@
//go:build !linux
package network
import (
"net"
"syscall"
)
func setTCPUserTimeout(conn syscall.RawConn, cfg net.KeepAliveConfig) {}
@@ -1,4 +1,5 @@
//go:build windows //go:build windows
// +build windows
package network package network
+1 -1
View File
@@ -66,7 +66,7 @@ func (suite *SocksProxyTestSuite) SetupSuite() {
require.NoError(suite.T(), err) require.NoError(suite.T(), err)
suite.authURL = parsed suite.authURL = parsed
suite.baseNetwork = network.New(nil, "mtg", 0, 0, 0, network.DefaultKeepAliveConfig) suite.baseNetwork = network.New(nil, "mtg", 0, 0, 0)
} }
func (suite *SocksProxyTestSuite) TestIncorrectSchema() { func (suite *SocksProxyTestSuite) TestIncorrectSchema() {