REPOSITORY / ScuroNeko/mtg

Compare commits

DIFF REPOSITORY

Compare commits

...
36 Commits
Author SHA1 Message Date
9seconds 8b3c622ea6 Merge remote-tracking branch 'origin/stable' into v2 2026-03-29 23:33:24 +02:00
9seconds 9d43c2d759 Merge remote-tracking branch 'origin/master' into stable 2026-03-29 23:33:06 +02:00
9seconds 0840c7e3e5 Update PGO 2026-03-29 23:31:41 +02:00
9seconds de48e177b1 Update depndencies 2026-03-29 23:16:06 +02:00
Sergei ArkhipovandGitHub 3ed09146b9 Merge pull request #422 from 9seconds/release-ci
Build release artifacts in CI
2026-03-29 23:15:28 +02:00
9seconds 018bd2fdc1 Run release build 2026-03-29 22:54:08 +02:00
Sergei ArkhipovandGitHub 0ad3a06863 Merge pull request #421 from 9seconds/mips-save-mem
Decrease a relay buffer size for MIPS devices
2026-03-29 22:09:25 +02:00
9seconds d3a090d6b4 Decrease a relay buffer size for MIPS devices 2026-03-29 21:42:26 +02:00
Sergei ArkhipovandGitHub 1725a0d721 Merge pull request #420 from dolonet/fix/telegram-relay-idle-timeout
fix: apply idle timeout to Telegram relay
2026-03-29 17:14:58 +02:00
Alexey Dolotov 87988326ab retry CI 2026-03-29 17:02:53 +03:00
Alexey Dolotov ec271baab0 fix: apply idle timeout to Telegram relay
Wrap both sides of the Telegram relay in connIdleTimeout,
same as already done for domain fronting in #416.

Without this, if a client disappears (network drop, battery dies),
the TCP connection stays formally alive and the goroutine in the
worker pool blocks on io.CopyBuffer indefinitely. Under mass client
disconnects this accumulates zombie goroutines.

Fixes #417
2026-03-29 16:59:30 +03:00
Sergei ArkhipovandGitHub 139db15e83 Merge pull request #419 from 9seconds/timers
Remove clock goroutine
2026-03-29 15:46:55 +02:00
9seconds 0c030646f9 Remove clock goroutine
This is a followup for https://github.com/9seconds/mtg/issues/412 it
makes sense to manage timers inplace instead of creating for new
goroutines: saves memory
2026-03-29 15:39:33 +02:00
Sergei ArkhipovandGitHub 822560bede Merge pull request #418 from dolonet/public-ip-config
Add public-ipv4/public-ipv6 config options
2026-03-29 13:52:07 +02:00
Sergei ArkhipovandGitHub 9917f61bc8 Merge pull request #415 from dolonet/fix/event-stream-32bit-index-panic
fix: prevent index out of range panic on 32-bit platforms
2026-03-29 13:51:12 +02:00
Sergei ArkhipovandGitHub 735466b90d Merge pull request #416 from dolonet/fix/domain-fronting-idle-timeout
fix: apply idle timeout to domain fronting relay
2026-03-29 13:50:45 +02:00
Sergei ArkhipovandGitHub 6b51de8305 Merge pull request #414 from dolonet/optimize-per-connection-overhead
Reduce per-connection memory overhead
2026-03-29 13:49:08 +02:00
Alexey Dolotov 7b62e06e36 Retry CI: flaky antireplay bloom filter test 2026-03-29 00:53:20 +03:00
Alexey Dolotov 2b07c0037e Add public-ipv4/public-ipv6 config options for manual IP override
On some servers ifconfig.co is unreachable (e.g. Hetzner, AdGuard DNS
blocklists), causing 'mtg doctor' SNI-DNS check and 'mtg access' link
generation to fail. New config options allow specifying public IPs
manually, with automatic detection as fallback.

Fixes #405
2026-03-29 00:47:49 +03:00
Alexey Dolotov 450381ee16 ci: retrigger (flaky antireplay test) 2026-03-28 23:13:35 +03:00
Alexey Dolotov 46c33f1532 ci: retrigger (flaky antireplay test) 2026-03-28 23:04:33 +03:00
Alexey Dolotov f355512aa6 fix: address staticcheck lint issues
- avoid deprecated DefaultIdleTimeout, use time.Minute directly
- simplify embedded field selectors (QF1008)
2026-03-28 22:59:32 +03:00
Alexey Dolotov 289bb283b1 fix: close connection on worker pool overflow
When the worker pool rejected a connection (ErrPoolOverload), the
accepted net.Conn was never closed — leaking a file descriptor and
TCP socket per rejected connection. Under sustained traffic spikes this
compounds the problem: leaked descriptors reduce the capacity for new
dials (including to the fronting domain), accelerating the failure
cascade described in #378.
2026-03-28 22:52:39 +03:00
Alexey Dolotov 836090ebdf fix: apply idle timeout to domain fronting relay connections
Domain fronting relay (for non-Telegram traffic) had no idle timeout,
causing worker pool exhaustion under traffic spikes.

The ProxyOpts.IdleTimeout field existed but was never wired into the
proxy. Now domain fronting connections are wrapped with per-read/write
deadlines reset to the configured idle timeout (default 1m), so stale
or slowloris-style connections are reaped promptly.

Fixes #378
2026-03-28 22:47:39 +03:00
Alexey Dolotov 01402bdba2 fix: prevent index out of range panic on 32-bit platforms
On 32-bit architectures (e.g. ARM7), int is 32 bits wide.
Casting a uint32 hash value to int can overflow, producing a
negative number. Go's modulo operator preserves the sign, so
the channel index can become -1, causing a panic.

Perform the modulo in uint32 space before indexing to ensure
the result is always non-negative.

Fixes #413
2026-03-28 22:31:47 +03:00
Alexey Dolotov 026ec74dfd Reduce per-connection memory overhead
- Use sync.Pool for relay buffers instead of stack-allocated arrays.
  A [16379]byte on the goroutine stack forces Go to grow it to 32KB
  (next power of two). Pooled buffers keep goroutine stacks small.

- Same fix for doppelganger write buffer ([16384]byte in conn.start).

- Replace idle goroutines with context.AfterFunc in proxy.ServeConn
  and relay.Relay. These goroutines existed only to wait on ctx.Done()
  and close connections. AfterFunc achieves the same without allocating
  a goroutine until the context is actually cancelled.

Net effect: at 3000 concurrent connections on a 1-vCPU/961MB VPS,
the unmodified binary drops 246 connections and falls to 10 MB/s.
With these changes: zero failures, 63 MB/s, 31% lower RSS.

Closes #412
2026-03-28 13:24:39 +03:00
Sergei ArkhipovandGitHub cc4b6ce2f4 Merge pull request #409 from dolonet/cert-noise-calibration
Add dynamic cert noise calibration for FakeTLS handshake
2026-03-28 09:04:18 +01:00
Alexey Dolotov 9dfd992c1d Move cert noise calibration into doppelganger scout
Instead of a separate cert_probe.go that duplicates the scout's TLS
connection logic, measure the cert chain size directly from the same
HTTPS connections the scout already makes.

Changes:
- Extend ScoutConnResult with payloadLen field
- Add Write interception to ScoutConn for handshake boundary detection
- Scout.learn() now computes cert size (sum of ApplicationData between
  CCS and first client Write) alongside inter-record durations
- Ganger aggregates cert sizes across raids and exposes NoiseParams()
  via atomic pointer for lock-free reads from proxy goroutines
- Proxy reads NoiseParams from Ganger on each handshake instead of
  probing at startup
- Remove cert_probe.go, disk cache, and related config options
  (noise-cache-path, noise-cache-ttl, noise-probe-count)

Falls back to legacy 2500-4700 range until the first scout raid
completes (typically within 1-2 seconds of startup).
2026-03-27 16:34:42 +03:00
Alexey Dolotov 80213ad35d Add dynamic cert noise calibration for FakeTLS handshake
The hardcoded noise range (2500-4700 bytes) in the FakeTLS ServerHello
does not match the real certificate chain sizes of many popular fronting
domains (e.g., dl.google.com ≈ 6480 bytes, microsoft.com ≈ 13004 bytes).
This makes the proxy detectable by DPI systems that compare the
ApplicationData size with the real cert chain size for the SNI domain.

On startup, probe the fronting domain's actual TLS handshake size and
use the measured value ± jitter instead of the static range. Falls back
to the legacy 2500-4700 range if the probe fails.

Also adds optional caching of probe results between restarts
(noise-cache-path, noise-cache-ttl) and a configurable probe count
(noise-probe-count) under [defense.doppelganger].

Closes #408
2026-03-26 23:38:58 +03:00
Sergei ArkhipovandGitHub d32e8e8b97 Merge pull request #404 from 9seconds/codeql
Update stale codeql configuration
2026-03-25 10:19:15 +01:00
9seconds 60c57c2306 Update stale codeql configuration 2026-03-25 10:17:30 +01:00
Sergei ArkhipovandGitHub 0edd5e6f92 Merge pull request #402 from 9seconds/PGO
Update PGO
2026-03-24 21:07:10 +01:00
9seconds a8e4acb6f8 Update PGO 2026-03-24 21:04:30 +01:00
Sergei ArkhipovandGitHub 27d10e6820 Merge pull request #401 from 9seconds/fix-prof
Fix build with profiling
2026-03-24 20:50:41 +01:00
9seconds b47e13556e Fix build with profiling 2026-03-24 20:46:41 +01:00
9seconds de81ed565d Add mention of fork 2026-03-24 15:28:33 +01:00
39 changed files with 483 additions and 240 deletions
+33
View File
@@ -119,6 +119,39 @@ jobs:
- name: Run linter
run: mise tasks run lint
artifacts:
name: Build release artifacts
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@v6
with:
submodules: recursive
- uses: jdx/mise-action@v3
name: Install mise
- name: Cache Go modules
uses: actions/cache@v5
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-gomod-${{ hashFiles('go.sum') }}
restore-keys: |
${{ runner.os }}-gomod-
- name: Cache cross-compilation build
uses: actions/cache@v5
with:
path: ~/.cache/go-build
key: ${{ runner.os }}-goreleaser-${{ hashFiles('go.sum') }}-${{ hashFiles('**/*.go') }}
restore-keys: |
${{ runner.os }}-goreleaser-${{ hashFiles('go.sum') }}-
${{ runner.os }}-goreleaser-
- name: Run release
run: mise tasks run release
docker:
name: Docker
runs-on: ubuntu-latest
+7 -5
View File
@@ -21,7 +21,7 @@ permissions:
on:
push:
branches:
branches:
- master
- stable
pull_request:
@@ -45,11 +45,13 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v6
with:
submodules: recursive
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -60,7 +62,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1
uses: github/codeql-action/autobuild@v4
# ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@@ -74,4 +76,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
uses: github/codeql-action/analyze@v4
+6
View File
@@ -16,6 +16,12 @@ sources = ["**/*.go", "go.mod", "go.sum"]
outputs = ["mtg"]
run = "go build"
[tasks."build:prof"]
description = "Build binary with profiling enabled"
sources = ["**/*.go", "go.mod", "go.sum"]
outputs = ["mtg"]
run = "go build -tags prof"
[tasks.update]
description = "Update dependencies"
run = [
+3
View File
@@ -91,6 +91,9 @@ that probably matter.
software. I also believe that in the case of throwout proxies, this
the feature is a useless luxury.
This is very controversial topic. Please read [rationale (in russian)](https://github.com/9seconds/mtg/issues/376#issuecomment-4118726699)
and use [mtg-multi](https://github.com/dolonet/mtg-multi) fork if you are disagree with.
* **No adtag support**
Please read [Version 2](#version-2) chapter.
BIN
View File
Binary file not shown.
+1 -1
View File
@@ -38,7 +38,7 @@ func (e EventStream) Send(ctx context.Context, evt mtglib.Event) {
select {
case <-ctx.Done():
case <-e.ctx.Done():
case e.chans[int(chanNo)%len(e.chans)] <- evt:
case e.chans[chanNo%uint32(len(e.chans))] <- evt:
}
}
+7
View File
@@ -48,6 +48,13 @@ concurrency = 8192
# Only ipv4 connectivity is used
prefer-ip = "prefer-ipv6"
# Public IP addresses of this server. Used by 'mtg access' to generate
# proxy links and by 'mtg doctor' to validate SNI-DNS match.
# If not set, mtg tries to detect them automatically via ifconfig.co.
# Set these if ifconfig.co is unreachable from your server.
# public-ipv4 = "1.2.3.4"
# public-ipv6 = "2001:db8::1"
# If this setting is set, then mtg will try to get proxy updates from Telegram
# Usually this is completely fine to have it disabled, because mtg has a list
# of some core proxies hardcoded.
+2 -2
View File
@@ -15,7 +15,7 @@ require (
github.com/prometheus/client_golang v1.23.2
github.com/prometheus/common v0.67.5 // indirect
github.com/prometheus/procfs v0.20.1 // indirect
github.com/rs/zerolog v1.34.0
github.com/rs/zerolog v1.35.0
github.com/smira/go-statsd v1.3.4
github.com/stretchr/objx v0.5.2 // indirect
github.com/stretchr/testify v1.11.1
@@ -29,7 +29,7 @@ require (
require (
github.com/beevik/ntp v1.5.0
github.com/ncruces/go-dns v1.3.2
github.com/pelletier/go-toml/v2 v2.2.4
github.com/pelletier/go-toml/v2 v2.3.0
github.com/pires/go-proxyproto v0.11.0
github.com/things-go/go-socks5 v0.1.0
github.com/txthinking/socks5 v0.0.0-20251011041537-5c31f201a10e
+4 -13
View File
@@ -18,14 +18,12 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/d4l3k/messagediff v1.2.1 h1:ZcAIMYsUg0EAp9X+tt8/enBE/Q8Yd5kzPynLyKptt9U=
github.com/d4l3k/messagediff v1.2.1/go.mod h1:Oozbb1TVXFac9FtSIxHBMnBCq2qeH/2KkEQxENCrlLo=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
@@ -40,11 +38,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mccutchen/go-httpbin v1.1.1 h1:aEws49HEJEyXHLDnshQVswfUlCVoS8g6h9YaDyaW7RE=
@@ -59,11 +54,10 @@ github.com/panjf2000/ants/v2 v2.12.0 h1:u9JhESo83i/GkZnhfTNuFMMWcNt7mnV1bGJ6FT4w
github.com/panjf2000/ants/v2 v2.12.0/go.mod h1:tSQuaNQ6r6NRhPt+IZVUevvDyFMTs+eS4ztZc52uJTY=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pelletier/go-toml/v2 v2.3.0 h1:k59bC/lIZREW0/iVaQR8nDHxVq8OVlIzYCOJf421CaM=
github.com/pelletier/go-toml/v2 v2.3.0/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pires/go-proxyproto v0.11.0 h1:gUQpS85X/VJMdUsYyEgyn59uLJvGqPhJV5YvG68wXH4=
github.com/pires/go-proxyproto v0.11.0/go.mod h1:ZKAAyp3cgy5Y5Mo4n9AlScrkCZwUy0g3Jf+slqQVcuU=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
@@ -76,9 +70,8 @@ github.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEy
github.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
github.com/rs/zerolog v1.34.0 h1:k43nTLIwcTVQAncfCw4KZ2VY6ukYoZaBPNOE8txlOeY=
github.com/rs/zerolog v1.34.0/go.mod h1:bJsvje4Z08ROH4Nhs5iH600c3IkWhwp44iRc54W6wYQ=
github.com/rs/zerolog v1.35.0 h1:VD0ykx7HMiMJytqINBsKcbLS+BJ4WYjz+05us+LRTdI=
github.com/rs/zerolog v1.35.0/go.mod h1:EjML9kdfa/RMA7h/6z6pYmq1ykOuA8/mjWaEvGI+jcw=
github.com/smira/go-statsd v1.3.4 h1:kBYWcLSGT+qC6JVbvfz48kX7mQys32fjDOPrfmsSx2c=
github.com/smira/go-statsd v1.3.4/go.mod h1:RjdsESPgDODtg1VpVVf9MJrEW2Hw0wtRNbmB1CAhu6A=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -133,10 +126,8 @@ golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+6
View File
@@ -58,6 +58,9 @@ func (a *Access) Run(cli *CLI, version string) error {
wg.Go(func() {
ip := a.PublicIPv4
if ip == nil {
ip = conf.PublicIPv4.Get(nil)
}
if ip == nil {
ip = getIP(ntw, "tcp4")
}
@@ -70,6 +73,9 @@ func (a *Access) Run(cli *CLI, version string) error {
})
wg.Go(func() {
ip := a.PublicIPv6
if ip == nil {
ip = conf.PublicIPv6.Get(nil)
}
if ip == nil {
ip = getIP(ntw, "tcp6")
}
+10 -3
View File
@@ -332,13 +332,20 @@ func (d *Doctor) checkSecretHost(resolver *net.Resolver, ntw mtglib.Network) boo
return false
}
ourIP4 := getIP(ntw, "tcp4")
ourIP6 := getIP(ntw, "tcp6")
ourIP4 := d.conf.PublicIPv4.Get(nil)
if ourIP4 == nil {
ourIP4 = getIP(ntw, "tcp4")
}
ourIP6 := d.conf.PublicIPv6.Get(nil)
if ourIP6 == nil {
ourIP6 = getIP(ntw, "tcp6")
}
if ourIP4 == nil && ourIP6 == nil {
tplError.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"description": "cannot detect public IP address",
"error": errors.New("ifconfig.co is unreachable for both IPv4 and IPv6"),
"error": errors.New("cannot detect automatically and public-ipv4/public-ipv6 are not set in config"),
})
return false
}
+2
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"net"
"os"
"time"
"github.com/9seconds/mtg/v2/antireplay"
"github.com/9seconds/mtg/v2/events"
@@ -262,6 +263,7 @@ func runProxy(conf *config.Config, version string) error { //nolint: funlen
AllowFallbackOnUnknownDC: conf.AllowFallbackOnUnknownDC.Get(false),
TolerateTimeSkewness: conf.TolerateTimeSkewness.Value,
IdleTimeout: conf.Network.Timeout.Idle.Get(time.Minute),
DoppelGangerURLs: doppelGangerURLs,
DoppelGangerPerRaid: conf.Defense.Doppelganger.Repeats.Get(mtglib.DoppelGangerPerRaid),
+6 -4
View File
@@ -35,6 +35,8 @@ type Config struct {
DomainFrontingProxyProtocol TypeBool `json:"domainFrontingProxyProtocol"`
TolerateTimeSkewness TypeDuration `json:"tolerateTimeSkewness"`
Concurrency TypeConcurrency `json:"concurrency"`
PublicIPv4 TypeIP `json:"publicIpv4"`
PublicIPv6 TypeIP `json:"publicIpv6"`
DomainFronting struct {
IP TypeIP `json:"ip"`
Port TypePort `json:"port"`
@@ -50,10 +52,10 @@ type Config struct {
Blocklist ListConfig `json:"blocklist"`
Allowlist ListConfig `json:"allowlist"`
Doppelganger struct {
URLs []TypeHttpsURL `json:"urls"`
Repeats TypeConcurrency `json:"repeats_per_raid"`
UpdateEach TypeDuration `json:"raid_each"`
DRS TypeBool `json:"drs"`
URLs []TypeHttpsURL `json:"urls"`
Repeats TypeConcurrency `json:"repeats_per_raid"`
UpdateEach TypeDuration `json:"raid_each"`
DRS TypeBool `json:"drs"`
} `json:"doppelganger"`
} `json:"defense"`
Network struct {
+26
View File
@@ -42,6 +42,32 @@ func (suite *ConfigTestSuite) TestParseMinimalConfig() {
suite.Equal("0.0.0.0:3128", conf.BindTo.String())
}
func (suite *ConfigTestSuite) TestParsePublicIP() {
conf, err := config.Parse(suite.ReadConfig("public_ip.toml"))
suite.NoError(err)
suite.Equal("203.0.113.1", conf.PublicIPv4.Get(nil).String())
suite.Equal("2001:db8::1", conf.PublicIPv6.Get(nil).String())
}
func (suite *ConfigTestSuite) TestParsePublicIPv4Only() {
conf, err := config.Parse(suite.ReadConfig("public_ip_v4_only.toml"))
suite.NoError(err)
suite.Equal("203.0.113.1", conf.PublicIPv4.Get(nil).String())
suite.Nil(conf.PublicIPv6.Get(nil))
}
func (suite *ConfigTestSuite) TestParsePublicIPInvalid() {
_, err := config.Parse(suite.ReadConfig("public_ip_invalid.toml"))
suite.Error(err)
}
func (suite *ConfigTestSuite) TestParsePublicIPNotSet() {
conf, err := config.Parse(suite.ReadConfig("minimal.toml"))
suite.NoError(err)
suite.Nil(conf.PublicIPv4.Get(nil))
suite.Nil(conf.PublicIPv6.Get(nil))
}
func (suite *ConfigTestSuite) TestString() {
conf, err := config.Parse(suite.ReadConfig("minimal.toml"))
suite.NoError(err)
+6 -4
View File
@@ -21,6 +21,8 @@ type tomlConfig struct {
DomainFrontingProxyProtocol bool `toml:"domain-fronting-proxy-protocol" json:"domainFrontingProxyProtocol,omitempty"`
TolerateTimeSkewness string `toml:"tolerate-time-skewness" json:"tolerateTimeSkewness,omitempty"`
Concurrency uint `toml:"concurrency" json:"concurrency,omitempty"`
PublicIPv4 string `toml:"public-ipv4" json:"publicIpv4,omitempty"`
PublicIPv6 string `toml:"public-ipv6" json:"publicIpv6,omitempty"`
DomainFronting struct {
IP string `toml:"ip" json:"ip,omitempty"`
Port uint `toml:"port" json:"port,omitempty"`
@@ -45,10 +47,10 @@ type tomlConfig struct {
UpdateEach string `toml:"update-each" json:"updateEach,omitempty"`
} `toml:"allowlist" json:"allowlist,omitempty"`
Doppelganger struct {
URLs []string `toml:"urls" json:"urls,omitempty"`
Repeats uint `toml:"repeats-per-raid" json:"repeats_per_raid,omitempty"`
UpdateEach string `toml:"raid-each" json:"raid_each,omitempty"`
DRS bool `toml:"drs" json:"drs,omitempty"`
URLs []string `toml:"urls" json:"urls,omitempty"`
Repeats uint `toml:"repeats-per-raid" json:"repeats_per_raid,omitempty"`
UpdateEach string `toml:"raid-each" json:"raid_each,omitempty"`
DRS bool `toml:"drs" json:"drs,omitempty"`
} `toml:"doppelganger" json:"doppelganger,omitempty"`
} `toml:"defense" json:"defense,omitempty"`
Network struct {
+4
View File
@@ -0,0 +1,4 @@
secret = "7oe1GqLy6TBc38CV3jx7q09nb29nbGUuY29t"
bind-to = "0.0.0.0:3128"
public-ipv4 = "203.0.113.1"
public-ipv6 = "2001:db8::1"
+3
View File
@@ -0,0 +1,3 @@
secret = "7oe1GqLy6TBc38CV3jx7q09nb29nbGUuY29t"
bind-to = "0.0.0.0:3128"
public-ipv4 = "not-an-ip"
+3
View File
@@ -0,0 +1,3 @@
secret = "7oe1GqLy6TBc38CV3jx7q09nb29nbGUuY29t"
bind-to = "0.0.0.0:3128"
public-ipv4 = "203.0.113.1"
+19
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"net"
"time"
"github.com/9seconds/mtg/v2/essentials"
"github.com/pires/go-proxyproto"
@@ -95,3 +96,21 @@ func newConnProxyProtocol(source, target essentials.Conn) *connProxyProtocol {
sourceAddr: source.RemoteAddr(),
}
}
type connIdleTimeout struct {
essentials.Conn
timeout time.Duration
}
func (c connIdleTimeout) Read(b []byte) (int, error) {
c.SetReadDeadline(time.Now().Add(c.timeout)) //nolint: errcheck
return c.Conn.Read(b) //nolint: wrapcheck
}
func (c connIdleTimeout) Write(b []byte) (int, error) {
c.SetWriteDeadline(time.Now().Add(c.timeout)) //nolint: errcheck
return c.Conn.Write(b) //nolint: wrapcheck
}
-35
View File
@@ -1,35 +0,0 @@
package doppel
import (
"context"
"time"
)
type Clock struct {
stats *Stats
tick chan struct{}
}
func (c Clock) Start(ctx context.Context) {
tickTock := time.NewTimer(c.stats.Delay())
defer func() {
tickTock.Stop()
select {
case <-tickTock.C:
default:
}
}()
for {
select {
case <-ctx.Done():
return
case <-tickTock.C:
select {
case <-ctx.Done():
case c.tick <- struct{}{}:
}
tickTock.Reset(c.stats.Delay())
}
}
}
-80
View File
@@ -1,80 +0,0 @@
package doppel
import (
"context"
"sync"
"testing"
"time"
"github.com/stretchr/testify/suite"
)
type ClockTestSuite struct {
suite.Suite
clock Clock
wg sync.WaitGroup
ctx context.Context
ctxCancel context.CancelFunc
}
func (suite *ClockTestSuite) SetupTest() {
ctx, cancel := context.WithCancel(context.Background())
suite.ctx = ctx
suite.ctxCancel = cancel
suite.clock = Clock{
stats: &Stats{
k: StatsDefaultK,
lambda: StatsDefaultLambda,
},
tick: make(chan struct{}),
}
suite.wg.Go(func() {
suite.clock.Start(suite.ctx)
})
}
func (suite *ClockTestSuite) TearDownTest() {
suite.ctxCancel()
suite.wg.Wait()
}
func (suite *ClockTestSuite) TestTicks() {
received := 0
for range 3 {
select {
case <-suite.clock.tick:
received++
case <-time.After(2 * time.Second):
suite.Fail("timed out waiting for tick")
}
}
suite.Equal(3, received)
}
func (suite *ClockTestSuite) TestStopsOnCancel() {
select {
case <-suite.clock.tick:
case <-time.After(2 * time.Second):
suite.Fail("timed out waiting for first tick")
}
suite.ctxCancel()
time.Sleep(50 * time.Millisecond)
select {
case <-suite.clock.tick:
suite.Fail("received tick after cancel")
default:
}
}
func TestClock(t *testing.T) {
t.Parallel()
suite.Run(t, &ClockTestSuite{})
}
+21 -19
View File
@@ -4,11 +4,19 @@ import (
"bytes"
"context"
"sync"
"time"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
var doppelBufPool = sync.Pool{
New: func() any {
b := make([]byte, tls.MaxRecordSize)
return &b
},
}
type Conn struct {
essentials.Conn
@@ -18,7 +26,7 @@ type Conn struct {
type connPayload struct {
ctx context.Context
ctxCancel context.CancelCauseFunc
clock Clock
stats Stats
wg sync.WaitGroup
writeStream bytes.Buffer
writtenCond sync.Cond
@@ -39,23 +47,23 @@ func (c Conn) Write(p []byte) (int, error) {
return len(p), context.Cause(c.p.ctx)
}
func (c Conn) Start() {
c.p.wg.Go(func() {
c.start()
})
}
func (c Conn) start() {
buf := [tls.MaxRecordSize]byte{}
bp := doppelBufPool.Get().(*[]byte)
buf := *bp
defer doppelBufPool.Put(bp)
timer := time.NewTimer(c.p.stats.Delay())
defer timer.Stop()
for {
select {
case <-c.p.ctx.Done():
return
case <-c.p.clock.tick:
case <-timer.C:
timer.Reset(c.p.stats.Delay())
}
size := c.p.clock.stats.Size()
size := c.p.stats.Size()
c.p.writtenCond.L.Lock()
for c.p.writeStream.Len() == 0 && !c.p.done {
@@ -68,7 +76,7 @@ func (c Conn) start() {
continue
}
if err := tls.WriteRecordInPlace(c.Conn, buf[:], n); err != nil {
if err := tls.WriteRecordInPlace(c.Conn, buf, n); err != nil {
c.p.ctxCancel(err)
return
}
@@ -86,28 +94,22 @@ func (c Conn) Stop() {
c.p.wg.Wait()
}
func NewConn(ctx context.Context, conn essentials.Conn, stats *Stats) Conn {
func NewConn(ctx context.Context, conn essentials.Conn, stats Stats) Conn {
ctx, cancel := context.WithCancelCause(ctx)
rv := Conn{
Conn: conn,
p: &connPayload{
ctx: ctx,
ctxCancel: cancel,
stats: stats,
writtenCond: sync.Cond{
L: &sync.Mutex{},
},
clock: Clock{
stats: stats,
tick: make(chan struct{}),
},
},
}
rv.p.writeStream.Grow(tls.DefaultBufferSize)
rv.p.wg.Go(func() {
rv.p.clock.Start(ctx)
})
rv.p.wg.Go(func() {
rv.start()
})
+2 -2
View File
@@ -63,7 +63,7 @@ func (suite *ConnTestSuite) TearDownTest() {
}
func (suite *ConnTestSuite) makeConn() Conn {
return NewConn(suite.ctx, suite.connMock, &Stats{
return NewConn(suite.ctx, suite.connMock, Stats{
k: 2.0,
lambda: 0.01,
})
@@ -152,7 +152,7 @@ func (suite *ConnTestSuite) TestStopDoesNotDeadlockWhenStartIsWaiting() {
ctx, cancel := context.WithCancel(suite.ctx)
defer cancel()
c := NewConn(ctx, suite.connMock, &Stats{
c := NewConn(ctx, suite.connMock, Stats{
k: 2.0,
lambda: 0.01,
})
+92 -10
View File
@@ -2,7 +2,9 @@ package doppel
import (
"context"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/9seconds/mtg/v2/essentials"
@@ -12,8 +14,22 @@ const (
DoppelGangerMaxDurations = 4096
DoppelGangerScoutRaidEach = 6 * time.Hour
DoppelGangerScoutRepeats = 10
MinCertSizesToCalculate = 3
)
// NoiseParams holds the measured cert chain size for FakeTLS noise calibration.
// If Mean is 0, the caller should use a legacy fallback.
type NoiseParams struct {
Mean int
Jitter int
}
type scoutRaidResult struct {
durations []time.Duration
certSizes []int
}
type gangerConnRequest struct {
ret chan<- Conn
payload essentials.Conn
@@ -31,8 +47,11 @@ type Ganger struct {
drs bool
stats *Stats
stats Stats
durations []time.Duration
certSizes []int
noiseParams atomic.Pointer[NoiseParams]
connRequests chan gangerConnRequest
}
@@ -48,6 +67,16 @@ func (g *Ganger) Run() {
})
}
// NoiseParams returns the current cert-size-based noise parameters.
// Returns zero-value NoiseParams if not yet measured (caller should use fallback).
func (g *Ganger) NoiseParams() NoiseParams {
if p := g.noiseParams.Load(); p != nil {
return *p
}
return NoiseParams{}
}
func (g *Ganger) NewConn(conn essentials.Conn) (Conn, error) {
rvChan := make(chan Conn)
req := gangerConnRequest{
@@ -81,10 +110,10 @@ func (g *Ganger) run() {
}
}()
scoutCollectedChan := make(chan []time.Duration)
scoutCollectedChan := make(chan scoutRaidResult)
currentScoutCollectedChan := scoutCollectedChan
updatedStatsChan := make(chan *Stats)
updatedStatsChan := make(chan Stats)
g.wg.Go(func() {
g.runScoutRaid(scoutCollectedChan)
@@ -94,18 +123,29 @@ func (g *Ganger) run() {
select {
case <-g.ctx.Done():
return
case durations := <-currentScoutCollectedChan:
g.durations = append(g.durations, durations...)
case result := <-currentScoutCollectedChan:
g.durations = append(g.durations, result.durations...)
if len(g.durations) > DoppelGangerMaxDurations {
copy(g.durations, g.durations[len(g.durations)-DoppelGangerMaxDurations:])
g.durations = g.durations[:DoppelGangerMaxDurations]
}
// Update cert sizes and recompute noise params.
g.certSizes = append(g.certSizes, result.certSizes...)
if len(g.certSizes) > DoppelGangerMaxDurations {
g.certSizes = g.certSizes[len(g.certSizes)-DoppelGangerMaxDurations:]
}
if len(g.certSizes) >= MinCertSizesToCalculate {
g.updateNoiseParams()
}
if len(g.durations) < MinDurationsToCalculate {
continue
}
durations := g.durations
currentScoutCollectedChan = nil
g.wg.Go(func() {
select {
@@ -129,8 +169,45 @@ func (g *Ganger) run() {
}
}
func (g *Ganger) runScoutRaid(rvChan chan<- []time.Duration) {
durations := []time.Duration{}
func (g *Ganger) updateNoiseParams() {
if len(g.certSizes) == 0 {
return
}
sum := 0
for _, s := range g.certSizes {
sum += s
}
mean := sum / len(g.certSizes)
maxDev := 0
for _, s := range g.certSizes {
d := s - mean
if d < 0 {
d = -d
}
if d > maxDev {
maxDev = d
}
}
if maxDev < 100 {
maxDev = 100
}
np := &NoiseParams{Mean: mean, Jitter: maxDev}
g.noiseParams.Store(np)
g.logger.Info(fmt.Sprintf(
"updated noise params: mean=%d jitter=%d samples=%d",
mean, maxDev, len(g.certSizes),
))
}
func (g *Ganger) runScoutRaid(rvChan chan<- scoutRaidResult) {
var result scoutRaidResult
for range g.scoutRaidRepeats {
learned, err := g.scout.Learn(g.ctx)
@@ -138,13 +215,18 @@ func (g *Ganger) runScoutRaid(rvChan chan<- []time.Duration) {
g.logger.WarningError("cannot learn", err)
continue
}
durations = append(durations, learned...)
result.durations = append(result.durations, learned.Durations...)
if learned.CertSize > 0 {
result.certSizes = append(result.certSizes, learned.CertSize)
}
}
select {
case <-g.ctx.Done():
return
case rvChan <- durations:
case rvChan <- result:
}
}
@@ -174,7 +256,7 @@ func NewGanger(
scoutRaidEach: scoutEach,
scoutRaidRepeats: scoutRepeats,
drs: drs,
stats: &Stats{
stats: Stats{
k: StatsDefaultK,
lambda: StatsDefaultLambda,
drs: drs,
+47 -12
View File
@@ -12,36 +12,46 @@ import (
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
// ScoutResult holds measurements from a single scout HTTP request.
type ScoutResult struct {
Durations []time.Duration
CertSize int // total ApplicationData bytes during TLS handshake; 0 if unknown
}
type Scout struct {
network Network
urls []string
}
func (s Scout) Learn(ctx context.Context) ([]time.Duration, error) {
var durations []time.Duration
func (s Scout) Learn(ctx context.Context) (ScoutResult, error) {
var combined ScoutResult
for _, url := range s.urls {
learned, err := s.learn(ctx, url)
if err != nil {
return nil, err
return ScoutResult{}, err
}
durations = append(durations, learned...)
combined.Durations = append(combined.Durations, learned.Durations...)
if learned.CertSize > 0 && combined.CertSize == 0 {
combined.CertSize = learned.CertSize
}
}
return durations, nil
return combined, nil
}
func (s Scout) learn(ctx context.Context, url string) ([]time.Duration, error) {
func (s Scout) learn(ctx context.Context, url string) (ScoutResult, error) {
client, results := s.makeClient()
if !strings.HasPrefix(url, "https://") {
return nil, fmt.Errorf("url %s must be https", url)
return ScoutResult{}, fmt.Errorf("url %s must be https", url)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
return ScoutResult{}, err
}
resp, err := client.Do(req)
@@ -52,10 +62,12 @@ func (s Scout) learn(ctx context.Context, url string) ([]time.Duration, error) {
}
if err != nil || len(results.data) == 0 {
return nil, err
return ScoutResult{}, err
}
durations := []time.Duration{}
var result ScoutResult
// Compute inter-record durations (existing logic).
lastTimestamp := time.Time{}
for i, v := range results.data {
@@ -71,11 +83,34 @@ func (s Scout) learn(ctx context.Context, url string) ([]time.Duration, error) {
}
}
durations = append(durations, v.timestamp.Sub(lastTimestamp))
result.Durations = append(result.Durations, v.timestamp.Sub(lastTimestamp))
lastTimestamp = v.timestamp
}
return durations, nil
// Compute cert size: sum of ApplicationData payload between CCS and
// the first client Write (which marks the end of server handshake).
seenCCS := false
boundary := results.writeIndex
if boundary < 0 {
boundary = len(results.data)
}
for i, v := range results.data {
if i >= boundary {
break
}
if v.recordType == tls.TypeChangeCipherSpec {
seenCCS = true
continue
}
if seenCCS && v.recordType == tls.TypeApplicationData {
result.CertSize += v.payloadLen
}
}
return result, nil
}
func (s Scout) makeClient() (*http.Client, *ScoutConnCollected) {
+17 -4
View File
@@ -14,9 +14,10 @@ type ScoutConn struct {
results *ScoutConnCollected
rawBuf *bytes.Buffer
seenCCS bool
}
func (s ScoutConn) Read(p []byte) (int, error) {
func (s *ScoutConn) Read(p []byte) (int, error) {
buf := &bytes.Buffer{}
for {
@@ -31,7 +32,11 @@ func (s ScoutConn) Read(p []byte) (int, error) {
return 0, err
}
s.results.Add(recordType)
if recordType == tls.TypeChangeCipherSpec {
s.seenCCS = true
}
s.results.Add(recordType, int(length))
s.rawBuf.Write([]byte{recordType})
s.rawBuf.Write(tls.TLSVersion[:])
@@ -45,11 +50,19 @@ func (s ScoutConn) Read(p []byte) (int, error) {
}
}
func NewScoutConn(conn essentials.Conn, results *ScoutConnCollected) ScoutConn {
func (s *ScoutConn) Write(p []byte) (int, error) {
if s.seenCCS {
s.results.MarkWrite()
}
return s.Conn.Write(p)
}
func NewScoutConn(conn essentials.Conn, results *ScoutConnCollected) *ScoutConn {
rawBuf := &bytes.Buffer{}
rawBuf.Grow(tls.MaxRecordSize)
return ScoutConn{
return &ScoutConn{
Conn: tls.New(conn, false, false),
results: results,
rawBuf: rawBuf,
+14 -3
View File
@@ -9,21 +9,32 @@ const (
type ScoutConnResult struct {
timestamp time.Time
recordType byte
payloadLen int
}
type ScoutConnCollected struct {
data []ScoutConnResult
data []ScoutConnResult
writeIndex int // index at which client first wrote post-handshake data; -1 if not set
}
func (s *ScoutConnCollected) Add(record byte) {
func (s *ScoutConnCollected) Add(record byte, payloadLen int) {
s.data = append(s.data, ScoutConnResult{
timestamp: time.Now(),
recordType: record,
payloadLen: payloadLen,
})
}
// MarkWrite records the current data length as the handshake boundary.
func (s *ScoutConnCollected) MarkWrite() {
if s.writeIndex < 0 {
s.writeIndex = len(s.data)
}
}
func NewScoutConnCollected() *ScoutConnCollected {
return &ScoutConnCollected{
data: make([]ScoutConnResult, 0, ScoutConnCollectedPreallocSize),
data: make([]ScoutConnResult, 0, ScoutConnCollectedPreallocSize),
writeIndex: -1,
}
}
@@ -14,7 +14,7 @@ type ScoutConnCollectedTestSuite struct {
func (suite *ScoutConnCollectedTestSuite) TestAddSingle() {
collected := NewScoutConnCollected()
collected.Add(tls.TypeApplicationData)
collected.Add(tls.TypeApplicationData, 100)
suite.Len(collected.data, 1)
suite.Equal(byte(tls.TypeApplicationData), collected.data[0].recordType)
@@ -23,13 +23,13 @@ func (suite *ScoutConnCollectedTestSuite) TestAddSingle() {
func (suite *ScoutConnCollectedTestSuite) TestAddTimestampsAreMonotonic() {
collected := NewScoutConnCollected()
collected.Add(tls.TypeApplicationData)
collected.Add(tls.TypeApplicationData, 100)
time.Sleep(time.Microsecond)
collected.Add(tls.TypeApplicationData)
collected.Add(tls.TypeApplicationData, 100)
time.Sleep(time.Microsecond)
collected.Add(tls.TypeApplicationData)
collected.Add(tls.TypeApplicationData, 100)
for i := 1; i < len(collected.data); i++ {
suite.True(collected.data[i].timestamp.After(collected.data[i-1].timestamp))
+2 -2
View File
@@ -22,9 +22,9 @@ func (suite *ScoutTestSuite) SetupSuite() {
}
func (suite *ScoutTestSuite) TestCollectResults() {
durations, err := suite.scout.Learn(suite.ctx)
result, err := suite.scout.Learn(suite.ctx)
suite.NoError(err)
suite.Less(3, len(durations))
suite.Less(3, len(result.Durations))
}
func (suite *ScoutTestSuite) TestCollectNothing() {
+2 -2
View File
@@ -112,7 +112,7 @@ func (d *Stats) Size() int {
return TLSRecordSizeMax
}
func NewStats(durations []time.Duration, drs bool) *Stats {
func NewStats(durations []time.Duration, drs bool) Stats {
n := float64(len(durations))
// in milliseconds
@@ -162,7 +162,7 @@ func NewStats(durations []time.Duration, drs bool) *Stats {
// λ = (Σxᵢᵏ / n)^(1/k)
lambda := math.Pow(sumXK/n, 1.0/k)
return &Stats{
return Stats{
k: k,
lambda: lambda,
drs: drs,
@@ -0,0 +1,13 @@
//go:build mips || mipsle
package relay
import "github.com/9seconds/mtg/v2/mtglib/internal/tls"
const (
// MIPS is quite short in resources, and usually it means that it will run
// on Microtiks, OpenWRT-based routers or similar hardware. I think it worth
// to sacrifice a number of read syscalls (read, CPU load) to shrink
// limited RAM resources.
bufPoolSize = tls.MaxRecordPayloadSize / 2
)
@@ -0,0 +1,9 @@
//go:build !mips && !mipsle
package relay
import "github.com/9seconds/mtg/v2/mtglib/internal/tls"
const (
bufPoolSize = tls.MaxRecordPayloadSize
)
+18
View File
@@ -0,0 +1,18 @@
package relay
import "sync"
var bufPool = sync.Pool{
New: func() any {
b := make([]byte, bufPoolSize)
return &b
},
}
func acquireBuffer() *[]byte {
return bufPool.Get().(*[]byte)
}
func releaseBuffer(p *[]byte) {
bufPool.Put(p)
}
+6 -6
View File
@@ -6,7 +6,6 @@ import (
"io"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
func Relay(ctx context.Context, log Logger, telegramConn, clientConn essentials.Conn) {
@@ -16,11 +15,11 @@ func Relay(ctx context.Context, log Logger, telegramConn, clientConn essentials.
ctx, cancel := context.WithCancel(ctx)
defer cancel()
go func() {
<-ctx.Done()
stop := context.AfterFunc(ctx, func() {
telegramConn.Close() //nolint: errcheck
clientConn.Close() //nolint: errcheck
}()
})
defer stop()
closeChan := make(chan struct{})
@@ -36,12 +35,13 @@ func Relay(ctx context.Context, log Logger, telegramConn, clientConn essentials.
}
func pump(log Logger, src, dst essentials.Conn, direction string) {
var buf [tls.MaxRecordPayloadSize]byte
buf := acquireBuffer()
defer releaseBuffer(buf)
defer src.CloseRead() //nolint: errcheck
defer dst.CloseWrite() //nolint: errcheck
n, err := io.CopyBuffer(src, dst, buf[:])
n, err := io.CopyBuffer(src, dst, *buf)
switch {
case err == nil:
+33 -14
View File
@@ -9,11 +9,18 @@ import (
"io"
rnd "math/rand/v2"
"github.com/9seconds/mtg/v2/mtglib/internal/doppel"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
"golang.org/x/crypto/curve25519"
)
// NoiseParams controls the size of the fake ApplicationData record
// in ServerHello. If Mean is 0, the legacy random range (2500-4700)
// is used.
type NoiseParams struct {
Mean int
Jitter int
}
const (
TypeHandshakeServer = 0x02
ChangeCipherValue = 0x01
@@ -33,13 +40,13 @@ var serverHelloSuffix = []byte{
0x00, 0x20, // 32 bytes of key
}
func SendServerHello(w io.Writer, secret []byte, clientHello *ClientHello) error {
func SendServerHello(w io.Writer, secret []byte, clientHello *ClientHello, noise NoiseParams) error {
buf := &bytes.Buffer{}
buf.Grow(tls.MaxRecordSize)
generateServerHello(buf, clientHello)
generateChangeCipherValue(buf)
generateNoise(buf)
generateNoise(buf, noise)
packet := buf.Bytes()
digest := hmac.New(sha256.New, secret)
@@ -125,19 +132,31 @@ func generateChangeCipherValue(buf *bytes.Buffer) {
buf.WriteByte(ChangeCipherValue)
}
func generateNoise(buf *bytes.Buffer) {
data := make(
[]byte,
int64(
doppel.TLSRecordSizeStart+rnd.IntN(
doppel.TLSRecordSizeAccel-doppel.TLSRecordSizeStart,
),
),
)
// generateNoise writes a single ApplicationData record mimicking the combined
// size of a real TLS 1.3 encrypted server handshake (EncryptedExtensions +
// Certificate chain + CertificateVerify + Finished).
//
// NOTE: Must be exactly ONE ApplicationData record — the Telegram client reads
// ServerHello + CCS + 1 ApplicationData and computes HMAC over all three.
// Multiple records would cause HMAC mismatch and connection failure.
func generateNoise(buf *bytes.Buffer, noise NoiseParams) {
var size int
if _, err := rand.Read(data[:]); err != nil {
if noise.Mean > 0 && noise.Jitter > 0 {
// Calibrated: use measured cert chain size ± jitter.
size = noise.Mean - noise.Jitter + rnd.IntN(2*noise.Jitter)
if size < 1000 {
size = 1000
}
} else {
// Legacy fallback: random in 2500-4700 range.
size = 2500 + rnd.IntN(2200)
}
data := make([]byte, size)
if _, err := rand.Read(data); err != nil {
panic(err)
}
tls.WriteRecord(buf, data[:]) //nolint: errcheck
tls.WriteRecord(buf, data) //nolint: errcheck
}
+32 -6
View File
@@ -8,7 +8,6 @@ import (
"testing"
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/doppel"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
"github.com/9seconds/mtg/v2/mtglib/internal/tls/fake"
"github.com/stretchr/testify/suite"
@@ -39,7 +38,7 @@ func (suite *SendServerHelloTestSuite) SetupTest() {
}
func (suite *SendServerHelloTestSuite) TestRecordStructure() {
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello)
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, fake.NoiseParams{})
suite.NoError(err)
var rec bytes.Buffer
@@ -59,13 +58,13 @@ func (suite *SendServerHelloTestSuite) TestRecordStructure() {
recordType, length, err := tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
suite.Equal(byte(tls.TypeApplicationData), recordType)
suite.Greater(length, int64(doppel.TLSRecordSizeStart))
suite.Greater(length, int64(2500))
suite.Empty(suite.buf.Bytes())
}
func (suite *SendServerHelloTestSuite) TestHMAC() {
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello)
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, fake.NoiseParams{})
suite.NoError(err)
packet := make([]byte, suite.buf.Len())
@@ -83,7 +82,7 @@ func (suite *SendServerHelloTestSuite) TestHMAC() {
}
func (suite *SendServerHelloTestSuite) TestHandshakePayload() {
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello)
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, fake.NoiseParams{})
suite.NoError(err)
packet := suite.buf.Bytes()
@@ -105,7 +104,7 @@ func (suite *SendServerHelloTestSuite) TestHandshakePayload() {
}
func (suite *SendServerHelloTestSuite) TestChangeCipherSpec() {
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello)
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, fake.NoiseParams{})
suite.NoError(err)
// Skip first record
@@ -124,6 +123,33 @@ func (suite *SendServerHelloTestSuite) TestChangeCipherSpec() {
suite.Equal([]byte{fake.ChangeCipherValue}, rec.Bytes())
}
func (suite *SendServerHelloTestSuite) TestCalibratedNoiseSize() {
noise := fake.NoiseParams{Mean: 6480, Jitter: 100}
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, noise)
suite.NoError(err)
var rec bytes.Buffer
// Skip ServerHello
_, _, err = tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
// Skip ChangeCipherSpec
rec.Reset()
_, _, err = tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
// Read noise ApplicationData
rec.Reset()
recordType, length, err := tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
suite.Equal(byte(tls.TypeApplicationData), recordType)
// Should be within mean ± jitter range.
suite.GreaterOrEqual(length, int64(noise.Mean-noise.Jitter))
suite.LessOrEqual(length, int64(noise.Mean+noise.Jitter))
}
func TestSendServerHello(t *testing.T) {
t.Parallel()
suite.Run(t, &SendServerHelloTestSuite{})
+14 -8
View File
@@ -27,6 +27,7 @@ type Proxy struct {
allowFallbackOnUnknownDC bool
tolerateTimeSkewness time.Duration
idleTimeout time.Duration
domainFrontingPort int
domainFrontingIP string
domainFrontingProxyProtocol bool
@@ -65,10 +66,10 @@ func (p *Proxy) ServeConn(conn essentials.Conn) {
ctx := newStreamContext(p.ctx, p.logger, conn)
defer ctx.Close()
go func() {
<-ctx.Done()
stop := context.AfterFunc(ctx, func() {
ctx.Close()
}()
})
defer stop()
p.eventStream.Send(ctx, NewEventStart(ctx.streamID, ctx.ClientIP()))
ctx.logger.Info("Stream has been started")
@@ -104,8 +105,8 @@ func (p *Proxy) ServeConn(conn essentials.Conn) {
relay.Relay(
ctx,
ctx.logger.Named("relay"),
ctx.telegramConn,
ctx.clientConn,
connIdleTimeout{Conn: ctx.telegramConn, timeout: p.idleTimeout},
connIdleTimeout{Conn: ctx.clientConn, timeout: p.idleTimeout},
)
}
@@ -151,6 +152,7 @@ func (p *Proxy) Serve(listener net.Listener) error {
case errors.Is(err, ants.ErrPoolClosed):
return nil
case errors.Is(err, ants.ErrPoolOverload):
conn.Close() //nolint: errcheck
logger.Info("connection was concurrency limited")
p.eventStream.Send(p.ctx, NewEventConcurrencyLimited())
}
@@ -192,7 +194,10 @@ func (p *Proxy) doFakeTLSHandshake(ctx *streamContext) bool {
return false
}
if err := fake.SendServerHello(ctx.clientConn, p.secret.Key[:], clientHello); err != nil {
gangerNoise := p.doppelGanger.NoiseParams()
noiseParams := fake.NoiseParams{Mean: gangerNoise.Mean, Jitter: gangerNoise.Jitter}
if err := fake.SendServerHello(ctx.clientConn, p.secret.Key[:], clientHello, noiseParams); err != nil {
p.logger.InfoError("cannot send welcome packet", err)
return false
}
@@ -303,8 +308,8 @@ func (p *Proxy) doDomainFronting(ctx *streamContext, conn *connRewind) {
relay.Relay(
ctx,
ctx.logger.Named("domain-fronting"),
frontConn,
conn,
connIdleTimeout{Conn: frontConn, timeout: p.idleTimeout},
connIdleTimeout{Conn: conn, timeout: p.idleTimeout},
)
}
@@ -336,6 +341,7 @@ func NewProxy(opts ProxyOpts) (*Proxy, error) {
domainFrontingPort: opts.getDomainFrontingPort(),
domainFrontingIP: opts.DomainFrontingIP,
tolerateTimeSkewness: opts.getTolerateTimeSkewness(),
idleTimeout: opts.getIdleTimeout(),
allowFallbackOnUnknownDC: opts.AllowFallbackOnUnknownDC,
telegram: tg,
doppelGanger: doppel.NewGanger(
+9
View File
@@ -160,6 +160,7 @@ type ProxyOpts struct {
// DoppelGangerDRS defines if TLS Dynamic Record Sizing is active.
DoppelGangerDRS bool
}
func (p ProxyOpts) valid() error {
@@ -215,6 +216,14 @@ func (p ProxyOpts) getPreferIP() string {
return p.PreferIP
}
func (p ProxyOpts) getIdleTimeout() time.Duration {
if p.IdleTimeout == 0 {
return time.Minute
}
return p.IdleTimeout
}
func (p ProxyOpts) getLogger(name string) Logger {
return p.Logger.Named(name)
}
-1
View File
@@ -3,7 +3,6 @@
package main
import (
"fmt"
"net"
"net/http"
_ "net/http/pprof" //nolint: gosec