REPOSITORY / ScuroNeko/mtg

Compare commits

DIFF REPOSITORY

Compare commits

..
186 Commits
Author SHA1 Message Date
9seconds 83a31e0458 Merge remote-tracking branch 'origin/stable' into v2 2026-04-07 18:10:41 +02:00
9seconds 95db895136 Merge remote-tracking branch 'origin/master' into stable 2026-04-07 18:10:26 +02:00
9seconds fb94d4a78b Update dependencies 2026-04-07 18:09:39 +02:00
Sergei ArkhipovandGitHub b5799500e4 Merge pull request #454 from 9seconds/tcp-notsent-lowat
Add TCP_NOTSENT_LOWAT setting
2026-04-07 15:45:53 +02:00
9seconds 437dacfaab Refactor socksopts per functionality, not per build flag 2026-04-07 15:42:22 +02:00
9seconds 0de8b28de8 Add TCP_NOTSENT_LOWAT setting 2026-04-07 15:39:26 +02:00
Sergei ArkhipovandGitHub 2f62e8055d Merge pull request #453 from 9seconds/tcp-user-timeout
Add TCP_USER_TIMEOUT support
2026-04-07 15:38:11 +02:00
9seconds b58ac669d5 Add TCP_USER_TIMEOUT support 2026-04-07 15:35:47 +02:00
Sergei ArkhipovandGitHub 60ab083def Merge pull request #452 from 9seconds/tcp-bbr
Use TCP BBR in a best-effort mode
2026-04-07 15:33:27 +02:00
9seconds 88f33debab Use TCP BBR in a best-effort mode
This small commits conditionally sets TCP BBR as a preferrable algorithm
for TCP congestion control
2026-04-07 14:54:56 +02:00
Sergei ArkhipovandGitHub ac40c94820 Merge pull request #451 from 9seconds/keepalive-config
Propagate keep alive settings from the config
2026-04-07 13:53:00 +02:00
9seconds 102f8a6cce Propagate keep alive settings from the config 2026-04-07 13:41:44 +02:00
Sergei ArkhipovandGitHub 83b43aecc1 Merge pull request #448 from 9seconds/no-default-tls-cipher
Do not use default TLS cipher
2026-04-07 12:13:21 +02:00
9seconds 5bf218f9ab Do not use default TLS cipher
As per RFC, if TLS server cannot pickup a suitable cipher from a client
list, it has to send handshake_failure alert. For us it means that we
have to route a request to a fronting domain, because we want to have it
exactly like a real webserver does. So, if it misbehaves, so do we.
2026-04-07 12:08:14 +02:00
Sergei ArkhipovandGitHub efc65f317e Merge pull request #447 from 9seconds/handshake-timeout
Add separate handshake timeout
2026-04-07 08:29:23 +02:00
9seconds 39ab5570a8 Small refactoring 2026-04-07 08:29:05 +02:00
9seconds eb564936c7 Add separate handshake timeout
This PR adds a new setting to the config: `network.timeout`. This setting
defines a time period during which all handshake procedures and
ceremonies must be completed. If not - connection is aborted. This
should help in situations when connection is established but client
cannot continue for some reason (for example, RST sent by some middle box).
2026-04-07 08:01:51 +02:00
Sergei ArkhipovandGitHub 74a81a986a Merge pull request #446 from runixer/fix/grease-cipher-suite
Fix DPI detection: replace GREASE cipher suite in FakeTLS ServerHello
2026-04-07 07:47:21 +02:00
Constantine bec321d190 Fix DPI detection: skip GREASE cipher suite in ClientHello parsing
Instead of echoing the first cipher suite from ClientHello (which is
often a GREASE value like 0x5a5a), iterate the list and pick the first
real cipher suite. This is what real TLS servers do per RFC 8701.

Production data shows two client profiles:
- 87% send GREASE first, then 0x1301 (TLS_AES_128_GCM_SHA256)
- 13% send 0xc02b first (TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256)

The fix correctly selects 0x1301 or 0xc02b respectively, matching
real server behavior. Fallback to 0x1301 if all suites are GREASE.

Add snapshot test with GREASE as first cipher suite.
2026-04-06 23:16:08 +03:00
Sergei ArkhipovandGitHub 45f958e527 Merge pull request #441 from appolimp/tcp-keepalive-idle-timeout
Improve TCP keepalive and idle timeout for mobile clients
2026-04-06 21:27:42 +02:00
appolimp 5f81ae3743 Improve TCP keepalive and idle timeout for mobile clients
TCP keepalive was configured (SetKeepAlivePeriod) but never actually
enabled (SO_KEEPALIVE) on accepted client connections. Go 1.26's
SetKeepAlivePeriod only sets TCP_KEEPIDLE — it does not call
setsockopt(SO_KEEPALIVE, 1). Without SO_KEEPALIVE the kernel never
sends probe packets, so dead connections from sleeping mobile clients
linger until the idle timeout fires.

Replace SetKeepAlive + SetKeepAlivePeriod with net.KeepAliveConfig
(available since Go 1.24) for explicit per-socket control:

  Idle:     30s   (time before first probe)
  Interval: 10s   (between probes)
  Count:    3     (failed probes to declare dead)

This detects dead connections in ~60s instead of relying on system
defaults (tcp_keepalive_intvl=75s, probes=9 → up to 11 minutes).

Increase the default idle timeout from 1 minute to 5 minutes.
MTProto clients send ping_delay_disconnect every ~60s, which resets
the idle timer. The previous 1-minute default created a race: if a
ping arrived even 1–2 seconds late the relay was killed. A 5-minute
window also survives typical mobile sleep periods (phone idle 2–5 min)
where the NAT mapping is still alive and the connection can resume
without reconnection.

Ref: #132
2026-04-04 12:01:33 +03:00
9seconds 1a81efcb6e Merge remote-tracking branch 'origin/stable' into v2 2026-04-01 17:17:22 +02:00
9seconds 2544c521ed Merge remote-tracking branch 'origin/master' into stable 2026-04-01 17:16:55 +02:00
9seconds 3a68ea5f2d Update goreleaser 2026-04-01 17:05:30 +02:00
Sergei ArkhipovandGitHub dbced77566 Merge pull request #433 from 9seconds/refactor-tls-fragmentation
Refactor TLS fragmenting
2026-04-01 14:30:21 +02:00
9seconds f4f969e702 Refactor TLS fragmenting 2026-04-01 14:01:24 +02:00
Sergei ArkhipovandGitHub e8368f7645 Merge pull request #431 from appolimp/tls-record-reassembly-pr
Support fragmented TLS handshake records
2026-04-01 09:33:13 +02:00
appolimp 38abee7d7f Support fragmented TLS handshake records
DPI bypass tools like ByeDPI fragment a single TLS record into multiple
records to evade censorship. This broke ReadClientHello because it
assumed the entire ClientHello arrives in one TLS record.

Add reassembleTLSHandshake that reads continuation records and
reconstructs a single TLS record before parsing and HMAC verification.
Per RFC 5246 Section 6.2.1, handshake messages may be fragmented
across multiple records — this is valid TLS behavior.
2026-04-01 09:05:24 +03:00
9seconds a3663fe8b5 Increase timeout for CI artifacts build 2026-03-31 22:13:42 +02:00
9seconds 2aa3321bd4 Add more forks 2026-03-31 19:03:48 +02:00
Sergei ArkhipovandGitHub 3793558c4c Merge pull request #430 from 9seconds/golang-idiomatic
More idiomatic Golang
2026-03-31 17:23:11 +02:00
9seconds b6427ee321 More idiomatic Golang 2026-03-31 15:07:01 +02:00
Sergei ArkhipovandGitHub 0c9fa5e710 Merge pull request #428 from 9seconds/auto-update-prio{
Change IP address set priority
2026-03-31 12:56:58 +02:00
9seconds 1fcec38aea Change IP address set priority
For a couple of releases we use collected IPs as a prioritized source
for connecting to Telegram. But apparently, they work way worse than it
should, and having connectivity to core ip ALWAYS gives better results.
Thus, this PR flips priorities, so users could have auto-update enabled
as a source of secondary addresses, not primary ones
2026-03-31 11:05:49 +02:00
Sergei ArkhipovandGitHub 89930631cf Merge pull request #426 from dolonet/fix/flaky-ci-race-and-bloom 2026-03-31 07:10:00 +02:00
dolonet eedee63143 Address review: use slices.Clone, simplify concurrent test
- Replace manual make+copy with slices.Clone in Snapshot()
- Remove redundant _ = len(data); Snapshot() call alone is
  sufficient to exercise the lock under -race
2026-03-30 16:17:51 +00:00
Alexey Dolotov 73c6a3aa37 fix: tighten ScoutConnCollected encapsulation and add concurrency test
- Move error check before Snapshot() to avoid unnecessary allocation
- Update existing tests to use Snapshot() instead of direct field access
- Add TestConcurrentAddSnapshot to explicitly exercise the mutex
2026-03-30 15:05:50 +03:00
Alexey Dolotov e54d9d60d3 fix: stabilize flaky CI tests
1. Add sync.Mutex to ScoutConnCollected to eliminate data race between
   Add()/MarkWrite() in readLoop and learn() iterating results.
   Introduce Snapshot() for safe read access.

2. Increase bloom filter test size from 500 to 100000 to prevent
   false negatives from random eviction in the stable bloom filter.

3. Use Require().NoError() in TestHTTPSRequest to prevent nil-pointer
   panic on resp.Body.Close() when the request fails.

Fixes #425
2026-03-30 14:50:32 +03:00
9seconds db2e6031a3 Merge remote-tracking branch 'origin/stable' into v2 2026-03-30 13:09:05 +02:00
9seconds 4b8da719ae Merge remote-tracking branch 'origin/master' into stable 2026-03-30 13:08:51 +02:00
9seconds a2de52f071 Update PGO 2026-03-30 13:08:32 +02:00
Sergei ArkhipovandGitHub b926a0590c Merge pull request #424 from dolonet/fix/relay-idle-timeout-shared-tracker
fix: use shared idle tracker for relay connections
2026-03-30 12:54:51 +02:00
Alexey Dolotov 58e8c8f982 ci: trigger tests 2026-03-30 13:51:17 +03:00
Alexey Dolotov 4642546b35 test: add idleTracker and connIdleTimeout tests
Cover shared idle tracker behavior:
- tracker lifecycle (new, idle after timeout, touch resets)
- read/write with data touches tracker
- read retries on timeout when tracker is not idle
- read closes on timeout when tracker is idle
- shared tracker prevents false timeout across directions
2026-03-30 13:21:51 +03:00
Alexey Dolotov 4627910238 fix: use shared idle tracker for relay connections
connIdleTimeout previously set per-direction deadlines independently.
During media downloads the client→telegram direction can be idle at the
application level while telegram→client is actively streaming data.
After IdleTimeout (default 1 min) the idle direction's ReadDeadline
fires, tearing down the entire relay and breaking media transfers.

Replace the per-direction timeout with a shared atomic timestamp that
both pump goroutines update on any successful Read or Write. When a
ReadDeadline fires on the idle direction, we check the shared tracker:
if the other direction was recently active, we retry instead of closing.
The connection is only torn down when both directions are idle for the
full timeout period.

This matches the documented IdleTimeout contract: "if we have any
message which will pass to either direction, a timer is reset."

Overhead: one atomic.Int64 (8 bytes) per connection pair, one
atomic.Store (~1 ns) per Read/Write with data, zero extra goroutines.

Fixes #423
2026-03-30 10:34:16 +03:00
9seconds 8b3c622ea6 Merge remote-tracking branch 'origin/stable' into v2 2026-03-29 23:33:24 +02:00
9seconds 9d43c2d759 Merge remote-tracking branch 'origin/master' into stable 2026-03-29 23:33:06 +02:00
9seconds 0840c7e3e5 Update PGO 2026-03-29 23:31:41 +02:00
9seconds de48e177b1 Update depndencies 2026-03-29 23:16:06 +02:00
Sergei ArkhipovandGitHub 3ed09146b9 Merge pull request #422 from 9seconds/release-ci
Build release artifacts in CI
2026-03-29 23:15:28 +02:00
9seconds 018bd2fdc1 Run release build 2026-03-29 22:54:08 +02:00
Sergei ArkhipovandGitHub 0ad3a06863 Merge pull request #421 from 9seconds/mips-save-mem
Decrease a relay buffer size for MIPS devices
2026-03-29 22:09:25 +02:00
9seconds d3a090d6b4 Decrease a relay buffer size for MIPS devices 2026-03-29 21:42:26 +02:00
Sergei ArkhipovandGitHub 1725a0d721 Merge pull request #420 from dolonet/fix/telegram-relay-idle-timeout
fix: apply idle timeout to Telegram relay
2026-03-29 17:14:58 +02:00
Alexey Dolotov 87988326ab retry CI 2026-03-29 17:02:53 +03:00
Alexey Dolotov ec271baab0 fix: apply idle timeout to Telegram relay
Wrap both sides of the Telegram relay in connIdleTimeout,
same as already done for domain fronting in #416.

Without this, if a client disappears (network drop, battery dies),
the TCP connection stays formally alive and the goroutine in the
worker pool blocks on io.CopyBuffer indefinitely. Under mass client
disconnects this accumulates zombie goroutines.

Fixes #417
2026-03-29 16:59:30 +03:00
Sergei ArkhipovandGitHub 139db15e83 Merge pull request #419 from 9seconds/timers
Remove clock goroutine
2026-03-29 15:46:55 +02:00
9seconds 0c030646f9 Remove clock goroutine
This is a followup for https://github.com/9seconds/mtg/issues/412 it
makes sense to manage timers inplace instead of creating for new
goroutines: saves memory
2026-03-29 15:39:33 +02:00
Sergei ArkhipovandGitHub 822560bede Merge pull request #418 from dolonet/public-ip-config
Add public-ipv4/public-ipv6 config options
2026-03-29 13:52:07 +02:00
Sergei ArkhipovandGitHub 9917f61bc8 Merge pull request #415 from dolonet/fix/event-stream-32bit-index-panic
fix: prevent index out of range panic on 32-bit platforms
2026-03-29 13:51:12 +02:00
Sergei ArkhipovandGitHub 735466b90d Merge pull request #416 from dolonet/fix/domain-fronting-idle-timeout
fix: apply idle timeout to domain fronting relay
2026-03-29 13:50:45 +02:00
Sergei ArkhipovandGitHub 6b51de8305 Merge pull request #414 from dolonet/optimize-per-connection-overhead
Reduce per-connection memory overhead
2026-03-29 13:49:08 +02:00
Alexey Dolotov 7b62e06e36 Retry CI: flaky antireplay bloom filter test 2026-03-29 00:53:20 +03:00
Alexey Dolotov 2b07c0037e Add public-ipv4/public-ipv6 config options for manual IP override
On some servers ifconfig.co is unreachable (e.g. Hetzner, AdGuard DNS
blocklists), causing 'mtg doctor' SNI-DNS check and 'mtg access' link
generation to fail. New config options allow specifying public IPs
manually, with automatic detection as fallback.

Fixes #405
2026-03-29 00:47:49 +03:00
Alexey Dolotov 450381ee16 ci: retrigger (flaky antireplay test) 2026-03-28 23:13:35 +03:00
Alexey Dolotov 46c33f1532 ci: retrigger (flaky antireplay test) 2026-03-28 23:04:33 +03:00
Alexey Dolotov f355512aa6 fix: address staticcheck lint issues
- avoid deprecated DefaultIdleTimeout, use time.Minute directly
- simplify embedded field selectors (QF1008)
2026-03-28 22:59:32 +03:00
Alexey Dolotov 289bb283b1 fix: close connection on worker pool overflow
When the worker pool rejected a connection (ErrPoolOverload), the
accepted net.Conn was never closed — leaking a file descriptor and
TCP socket per rejected connection. Under sustained traffic spikes this
compounds the problem: leaked descriptors reduce the capacity for new
dials (including to the fronting domain), accelerating the failure
cascade described in #378.
2026-03-28 22:52:39 +03:00
Alexey Dolotov 836090ebdf fix: apply idle timeout to domain fronting relay connections
Domain fronting relay (for non-Telegram traffic) had no idle timeout,
causing worker pool exhaustion under traffic spikes.

The ProxyOpts.IdleTimeout field existed but was never wired into the
proxy. Now domain fronting connections are wrapped with per-read/write
deadlines reset to the configured idle timeout (default 1m), so stale
or slowloris-style connections are reaped promptly.

Fixes #378
2026-03-28 22:47:39 +03:00
Alexey Dolotov 01402bdba2 fix: prevent index out of range panic on 32-bit platforms
On 32-bit architectures (e.g. ARM7), int is 32 bits wide.
Casting a uint32 hash value to int can overflow, producing a
negative number. Go's modulo operator preserves the sign, so
the channel index can become -1, causing a panic.

Perform the modulo in uint32 space before indexing to ensure
the result is always non-negative.

Fixes #413
2026-03-28 22:31:47 +03:00
Alexey Dolotov 026ec74dfd Reduce per-connection memory overhead
- Use sync.Pool for relay buffers instead of stack-allocated arrays.
  A [16379]byte on the goroutine stack forces Go to grow it to 32KB
  (next power of two). Pooled buffers keep goroutine stacks small.

- Same fix for doppelganger write buffer ([16384]byte in conn.start).

- Replace idle goroutines with context.AfterFunc in proxy.ServeConn
  and relay.Relay. These goroutines existed only to wait on ctx.Done()
  and close connections. AfterFunc achieves the same without allocating
  a goroutine until the context is actually cancelled.

Net effect: at 3000 concurrent connections on a 1-vCPU/961MB VPS,
the unmodified binary drops 246 connections and falls to 10 MB/s.
With these changes: zero failures, 63 MB/s, 31% lower RSS.

Closes #412
2026-03-28 13:24:39 +03:00
Sergei ArkhipovandGitHub cc4b6ce2f4 Merge pull request #409 from dolonet/cert-noise-calibration
Add dynamic cert noise calibration for FakeTLS handshake
2026-03-28 09:04:18 +01:00
Alexey Dolotov 9dfd992c1d Move cert noise calibration into doppelganger scout
Instead of a separate cert_probe.go that duplicates the scout's TLS
connection logic, measure the cert chain size directly from the same
HTTPS connections the scout already makes.

Changes:
- Extend ScoutConnResult with payloadLen field
- Add Write interception to ScoutConn for handshake boundary detection
- Scout.learn() now computes cert size (sum of ApplicationData between
  CCS and first client Write) alongside inter-record durations
- Ganger aggregates cert sizes across raids and exposes NoiseParams()
  via atomic pointer for lock-free reads from proxy goroutines
- Proxy reads NoiseParams from Ganger on each handshake instead of
  probing at startup
- Remove cert_probe.go, disk cache, and related config options
  (noise-cache-path, noise-cache-ttl, noise-probe-count)

Falls back to legacy 2500-4700 range until the first scout raid
completes (typically within 1-2 seconds of startup).
2026-03-27 16:34:42 +03:00
Alexey Dolotov 80213ad35d Add dynamic cert noise calibration for FakeTLS handshake
The hardcoded noise range (2500-4700 bytes) in the FakeTLS ServerHello
does not match the real certificate chain sizes of many popular fronting
domains (e.g., dl.google.com ≈ 6480 bytes, microsoft.com ≈ 13004 bytes).
This makes the proxy detectable by DPI systems that compare the
ApplicationData size with the real cert chain size for the SNI domain.

On startup, probe the fronting domain's actual TLS handshake size and
use the measured value ± jitter instead of the static range. Falls back
to the legacy 2500-4700 range if the probe fails.

Also adds optional caching of probe results between restarts
(noise-cache-path, noise-cache-ttl) and a configurable probe count
(noise-probe-count) under [defense.doppelganger].

Closes #408
2026-03-26 23:38:58 +03:00
Sergei ArkhipovandGitHub d32e8e8b97 Merge pull request #404 from 9seconds/codeql
Update stale codeql configuration
2026-03-25 10:19:15 +01:00
9seconds 60c57c2306 Update stale codeql configuration 2026-03-25 10:17:30 +01:00
Sergei ArkhipovandGitHub 0edd5e6f92 Merge pull request #402 from 9seconds/PGO
Update PGO
2026-03-24 21:07:10 +01:00
9seconds a8e4acb6f8 Update PGO 2026-03-24 21:04:30 +01:00
Sergei ArkhipovandGitHub 27d10e6820 Merge pull request #401 from 9seconds/fix-prof
Fix build with profiling
2026-03-24 20:50:41 +01:00
9seconds b47e13556e Fix build with profiling 2026-03-24 20:46:41 +01:00
9seconds de81ed565d Add mention of fork 2026-03-24 15:28:33 +01:00
9seconds 006fba1046 Merge remote-tracking branch 'origin/stable' into v2 2026-03-24 09:59:03 +01:00
9seconds 7b333ed833 Merge remote-tracking branch 'origin/master' into stable 2026-03-24 09:58:48 +01:00
9seconds 5adfee5dd4 Remove wrong binary 2026-03-24 09:58:13 +01:00
9seconds de89de2ad6 Merge remote-tracking branch 'origin/master' into stable 2026-03-24 09:57:43 +01:00
9seconds b0d37de0ec Update linter 2026-03-24 09:57:20 +01:00
9seconds 0cb25ba7ff Update go dependencies 2026-03-24 09:55:53 +01:00
9seconds 614acd7303 Mention doctor in README 2026-03-24 09:55:30 +01:00
Sergei ArkhipovandGitHub 4f5368aa2a Merge pull request #398 from 9seconds/docker-directory
Allow using directory bind mounts for a docker container
2026-03-24 09:00:59 +01:00
9seconds cfb5fe66be Allow using directory bind mounts for a docker container
This helps with a situation when some applications do not allow mounting
individual files, but whole directories. In that case users could mount
`/config` directory with a single file, `config.toml`: `-v
/path/to/dir:/config`. Also, there is a backward compatibility to using
a single `/config.toml`
2026-03-24 08:48:42 +01:00
Sergei ArkhipovandGitHub fb390d3417 Merge pull request #397 from 9seconds/doctor 2026-03-23 19:35:49 +01:00
9seconds f0ae4ce290 Validate domain fronting availability 2026-03-23 19:22:21 +01:00
9seconds b6b900e430 Refactoring 2026-03-23 19:12:14 +01:00
9seconds 8154f65e0e Add validation of telegram connectivity 2026-03-23 18:34:38 +01:00
9seconds a60523fed0 Add verification of time skewness 2026-03-23 15:28:48 +01:00
9seconds 63b147c287 Add doctor command for deprecated config values 2026-03-23 14:45:10 +01:00
Sergei ArkhipovandGitHub 21c0d18c7c Merge pull request #395 from roman901/master 2026-03-21 23:31:24 +01:00
Roman Shishkin 8f0bf47d56 Add Config.GetConcurrency with default fallback 2026-03-21 21:53:38 +03:00
9seconds 7fec30908a Merge remote-tracking branch 'origin/stable' into v2 2026-03-20 11:29:39 +01:00
9seconds 2eb0828f72 Merge remote-tracking branch 'origin/master' into stable 2026-03-20 11:29:24 +01:00
Sergei ArkhipovandGitHub d01e089f54 Merge pull request #386 from 9seconds/architectures
Add more architectures for mtg
2026-03-20 11:22:58 +01:00
Sergei ArkhipovandGitHub c736881792 Merge pull request #388 from 9seconds/doc-limits
Document a necessety of increasing limits for systemd unit
2026-03-20 11:16:06 +01:00
9seconds d5a118f125 Remove explicit pgo 2026-03-20 11:15:01 +01:00
9seconds d79a8f8406 Fix failed builds 2026-03-20 11:14:25 +01:00
9seconds 97932758d1 Add mips support 2026-03-20 11:14:25 +01:00
9seconds 1f7d1c0eea Add windows builds 2026-03-20 11:14:25 +01:00
9seconds 8c73dde928 Add build for AMD64v3 2026-03-20 11:14:25 +01:00
9seconds ded3fe26b9 Build for ARMv9 2026-03-20 11:14:25 +01:00
Sergei ArkhipovandGitHub 2f00adfe91 Merge pull request #385 from 9seconds/pgo
Add PGO
2026-03-20 11:14:00 +01:00
9seconds 049bee3d84 Document a necessety of increasing limits for systemd unit
It seems that default DynamicUser limits are very low. We have to
increase them anyway.
2026-03-20 11:13:03 +01:00
9seconds 4fbabfda2a Add PGO 2026-03-20 10:54:30 +01:00
9seconds fc72de9e39 Merge remote-tracking branch 'origin/stable' into v2 2026-03-19 18:52:35 +01:00
9seconds cb627f2a66 Merge remote-tracking branch 'origin/master' into stable 2026-03-19 18:52:11 +01:00
Sergei ArkhipovandGitHub 9ba6df0d1c Merge pull request #383 from 9seconds/avoid-double-buffering
Avoid double buffering in TLS hot path
2026-03-19 17:46:36 +01:00
9seconds 4a8d099aca Remove unused buffer 2026-03-19 17:39:57 +01:00
9seconds feb57004e1 Fix reslicing 2026-03-19 17:39:48 +01:00
9seconds cb436efd87 Avoid double buffering in TLS hot path 2026-03-19 17:37:51 +01:00
Sergei ArkhipovandGitHub 24148ea95c Merge pull request #382 from 9seconds/write-cond
Optimize waiting time for TLS chunker
2026-03-19 15:51:11 +01:00
9seconds 724904f50d Wait in doppel.Conn if there is anything to write 2026-03-19 15:42:00 +01:00
9seconds a23ae05f3b Remove SyncWrite 2026-03-19 13:47:08 +01:00
Sergei ArkhipovandGitHub b153a55149 Merge pull request #379 from 9seconds/fix-telegram-ips
Show ip of telegram endpoints in event stream
2026-03-18 22:46:47 +01:00
9seconds 913a38d13a Show real IP of the telegram endpoint in event stream 2026-03-18 22:05:34 +01:00
9seconds dc81f7981c Merge remote-tracking branch 'origin/stable' into v2 2026-03-16 23:56:10 +01:00
9seconds 9d5fd989e5 Merge remote-tracking branch 'origin/master' into stable 2026-03-16 23:55:56 +01:00
Sergei ArkhipovandGitHub 81703233b0 Merge pull request #368 from 9seconds/flake-tests
Fix flaky test
2026-03-16 23:55:01 +01:00
9seconds eb7720b11e Fix flaky test 2026-03-16 23:44:06 +01:00
Sergei ArkhipovandGitHub df7ddc3d6a Merge pull request #367 from saleacy/patch-1
fix: ensure network.Dial and MakeHTTPClient use socks5 proxy
2026-03-16 23:38:43 +01:00
saleacyandGitHub 3bc1e415f9 fix: ensure network.Dial and MakeHTTPClient use socks5 proxy
The package `network/v2/proxy_network.go` does not wrap `network.Dial`
and `network.MakeHTTPClient`, which causes them to bypass the SOCKS5
proxy and initiate TCP connections directly from the local machine.
2026-03-17 01:35:18 +08:00
Sergei ArkhipovandGitHub 306fa19ad6 Merge pull request #366 from Maks-2012/patch-1
Fix preferIPOnlyIPv6
2026-03-16 15:31:09 +01:00
Maks-2012andGitHub 079252d810 Fix preferIPOnlyIPv6 2026-03-16 16:10:38 +03:00
9seconds d0502e7083 Merge remote-tracking branch 'origin/stable' into v2 2026-03-15 22:04:35 +01:00
9seconds 30aa9d3a44 Merge remote-tracking branch 'origin/master' into stable 2026-03-15 22:04:09 +01:00
9seconds d98d5be3a7 Update go dependencies 2026-03-15 22:01:04 +01:00
9seconds 94ac9d6ffa Update tooling 2026-03-15 22:00:08 +01:00
9seconds c036558df4 Update docs 2026-03-15 21:58:04 +01:00
Sergei ArkhipovandGitHub 4167dec5e1 Merge pull request #365 from 9seconds/best-practices
Add document about best practices
2026-03-15 21:52:16 +01:00
9seconds 57529904e3 Add document about best practices 2026-03-15 21:50:43 +01:00
Sergei ArkhipovandGitHub ed4e511560 Merge pull request #363 from 9seconds/relay-buffer
Decrease relay buffer size
2026-03-14 14:15:52 +01:00
9seconds bb90bcc127 Decrease relay buffer size
Even if it makes sense to have a huge buffers, we do artificial delays
now. In that case we could achieve the same results with a lower buffer.
If not, then we won't send a packet bigger that this value
2026-03-14 14:06:48 +01:00
9seconds 879ebd132d Formatting of the README 2026-03-14 09:44:07 +01:00
Sergei ArkhipovandGitHub 58e6dfd603 Merge pull request #361 from 9seconds/docs
Add docs about doppelganger
2026-03-14 09:43:19 +01:00
9seconds e675baa860 Add docs about doppelganger 2026-03-14 09:42:28 +01:00
Sergei ArkhipovandGitHub 4a17849763 Merge pull request #360 from 9seconds/fix-android
Have a noise of variable size
2026-03-13 17:11:46 +01:00
9seconds 7adfc0352e Have a noise of variable size 2026-03-13 17:08:44 +01:00
Sergei ArkhipovandGitHub a5d602b538 Merge pull request #359 from 9seconds/fix-android
Fix android ping
2026-03-13 16:46:18 +01:00
9seconds b35c212d65 Have a noise of variable size 2026-03-13 16:39:16 +01:00
9seconds 45b0964afd Fix android ping 2026-03-13 16:20:18 +01:00
Sergei ArkhipovandGitHub 6d8d2961e8 Merge pull request #358 from 9seconds/doppel-drs
Make DRS optional
2026-03-13 14:51:15 +01:00
9seconds 4138cc6494 Use SyncWrite call to send a packet to proxy 2026-03-13 14:44:56 +01:00
9seconds 33c0fa9bf7 Add SyncWrite method to doppel.Conn 2026-03-13 11:30:52 +01:00
9seconds 21d7522356 Make DRS optional 2026-03-13 11:04:01 +01:00
9seconds ea71fe81b2 Update stats defaults 2026-03-13 08:59:59 +01:00
Sergei ArkhipovandGitHub 0871c2d790 Merge pull request #357 from 9seconds/access-native-dialer
Access command should use native dialer instead of network one
2026-03-13 08:47:24 +01:00
9seconds 8df1629882 Access command should use native dialer instead of network one 2026-03-13 08:42:15 +01:00
9seconds ad945c77a9 Enable caching for govulncheck 2026-03-12 23:01:26 +01:00
Sergei ArkhipovandGitHub a051518def Merge pull request #356 from 9seconds/gh-cache
Speed up Github actions
2026-03-12 22:51:59 +01:00
9seconds 4dca1d2b07 Add cache mount for apk downloads 2026-03-12 22:46:19 +01:00
9seconds 287a794772 Always run tests with race detector 2026-03-12 22:22:09 +01:00
9seconds d4822989ec Add count number to covtest 2026-03-12 22:22:00 +01:00
Sergei ArkhipovandGitHub d6a17c502f Merge pull request #355 from 9seconds/scout-native
Use native dialer for scout
2026-03-12 22:14:52 +01:00
9seconds 3db1be0687 Use cache to speed up jobs 2026-03-12 22:13:17 +01:00
9seconds 991346621c Speedup docker builds by using cache 2026-03-12 22:13:00 +01:00
9seconds cc63e762e3 Use native dialer for scout 2026-03-12 21:47:03 +01:00
9seconds a85348d6be Use curl as user agent 2026-03-12 21:43:04 +01:00
Sergei ArkhipovandGitHub 7762cf3a55 Merge pull request #353 from 9seconds/domain-fronting-native
Use native dialer to communicate with fronting domain
2026-03-12 21:40:38 +01:00
9seconds 62cba24071 Use native dialer to communicate with fronting domain 2026-03-12 20:51:02 +01:00
9seconds 94e4179fb7 Add NativeDialer method to mtglib.Network 2026-03-12 19:25:53 +01:00
Sergei ArkhipovandGitHub 6493688282 Merge pull request #352 from 9seconds/doppleganger
Doppleganger
2026-03-12 19:18:44 +01:00
9seconds ffd3ab03cc Formatting with gofumpt 2026-03-12 19:13:33 +01:00
9seconds 00403e3a94 Remove old faketls package 2026-03-12 19:11:39 +01:00
9seconds 0bfc1ef2d4 Fix double TLS wrapping for noise 2026-03-12 19:07:11 +01:00
9seconds 7aa01dcebe Limit min number of collected stats 2026-03-12 19:07:11 +01:00
9seconds 1c0e847247 Set statistics measured from real life 2026-03-12 19:07:11 +01:00
9seconds 83ff4ee266 Align scout defaults with config defaults 2026-03-12 19:07:11 +01:00
9seconds 7a58c74cfe Correctly process doppel conns 2026-03-12 19:07:11 +01:00
9seconds d43d6692d7 Align naming about scout raids 2026-03-12 19:07:11 +01:00
9seconds 75392941da Propagate settings to Proxy 2026-03-12 19:07:11 +01:00
9seconds 25ad776b6f Propagate doppelganger to config 2026-03-12 19:07:11 +01:00
9seconds 5557393b38 Add config section 2026-03-12 19:07:11 +01:00
9seconds bb49c6a55d Integrate new fake package and doppel into proxy 2026-03-12 19:07:11 +01:00
9seconds 37f8d18be5 Add server side of things 2026-03-12 19:07:11 +01:00
9seconds 59557059df ReadClientHello function 2026-03-12 19:07:10 +01:00
9seconds 1182b9ef6f Add doppel and tls packages 2026-03-12 19:07:10 +01:00
9seconds c886ffdd81 Add TrustTLS method to networkHTTPTransport 2026-03-11 23:21:41 +01:00
Sergei ArkhipovandGitHub e9b38a5b74 Merge pull request #348 from 9seconds/bundle-config 2026-03-05 19:09:48 +01:00
9seconds 036b10be67 Include example config into distribution bundle 2026-03-05 17:14:52 +01:00
135 changed files with 6060 additions and 1587 deletions
+3
View File
@@ -0,0 +1,3 @@
# git config merge.theirs.name "Always accept theirs"
# git config merge.theirs.driver "cp %B %A"
default.pgo binary merge=theirs
+66 -11
View File
@@ -48,6 +48,16 @@ jobs:
- uses: jdx/mise-action@v3
name: Install mise
- name: Cache Go modules and build
uses: actions/cache@v5
with:
path: |
~/go/pkg/mod
~/.cache/go-build
key: ${{ runner.os }}-go-${{ hashFiles('go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Run tests
run: mise tasks run covtest
@@ -69,6 +79,16 @@ jobs:
- uses: jdx/mise-action@v3
name: Install mise
- name: Cache Go modules and build
uses: actions/cache@v5
with:
path: |
~/go/pkg/mod
~/.cache/go-build
key: ${{ runner.os }}-go-${{ hashFiles('go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Run fuzzing
run: mise tasks run 'test:fuzz:*'
@@ -86,9 +106,52 @@ jobs:
- uses: jdx/mise-action@v3
name: Install mise
- name: Cache Go modules and build
uses: actions/cache@v5
with:
path: |
~/go/pkg/mod
~/.cache/go-build
key: ${{ runner.os }}-go-${{ hashFiles('go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Run linter
run: mise tasks run lint
artifacts:
name: Build release artifacts
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v6
with:
submodules: recursive
- uses: jdx/mise-action@v3
name: Install mise
- name: Cache Go modules
uses: actions/cache@v5
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-gomod-${{ hashFiles('go.sum') }}
restore-keys: |
${{ runner.os }}-gomod-
- name: Cache cross-compilation build
uses: actions/cache@v5
with:
path: ~/.cache/go-build
key: ${{ runner.os }}-goreleaser-${{ hashFiles('go.sum') }}-${{ hashFiles('**/*.go') }}
restore-keys: |
${{ runner.os }}-goreleaser-${{ hashFiles('go.sum') }}-
${{ runner.os }}-goreleaser-
- name: Run release
run: mise tasks run release
docker:
name: Docker
runs-on: ubuntu-latest
@@ -123,14 +186,6 @@ jobs:
- name: Setup BuildX
uses: docker/setup-buildx-action@v3
- name: Setup cache
uses: actions/cache@v5
with:
path: /tmp/buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
- name: Login to DockerHub
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
@@ -147,7 +202,7 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v2
uses: docker/build-push-action@v6
with:
pull: true
context: .
@@ -155,5 +210,5 @@ jobs:
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=local,src=/tmp/buildx-cache
cache-to: type=local,dest=/tmp/buildx-cache
cache-from: type=gha
cache-to: type=gha,mode=max
+7 -5
View File
@@ -21,7 +21,7 @@ permissions:
on:
push:
branches:
branches:
- master
- stable
pull_request:
@@ -45,11 +45,13 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v6
with:
submodules: recursive
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -60,7 +62,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1
uses: github/codeql-action/autobuild@v4
# ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@@ -74,4 +76,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
uses: github/codeql-action/analyze@v4
+1
View File
@@ -35,6 +35,7 @@ jobs:
uses: actions/setup-go@v6
with:
go-version-file: go.mod
cache: true
- name: Check for vulnerabilities
run: |
+83 -2
View File
@@ -10,13 +10,15 @@ before:
- go generate ./...
builds:
- binary: '{{ .ProjectName }}'
- id: default
binary: '{{ .ProjectName }}'
goos:
- darwin
- freebsd
- linux
- netbsd
- openbsd
- windows
goarch:
- 386
- amd64
@@ -34,15 +36,74 @@ builds:
ignore:
- goos: darwin
goarch: 386
- goos: darwin
goarch: arm
- goos: freebsd
goarch: arm64
- goos: netbsd
goarch: arm64
- goos: openbsd
goarch: arm64
- goos: windows
goarch: 386
- goos: windows
goarch: arm
- id: mips
binary: '{{ .ProjectName }}'
goos:
- linux
goarch:
- mips
- mipsle
gomips:
- softfloat
env:
- CGO_ENABLED=0
flags:
- -trimpath
- -mod=readonly
ldflags: -s -w -X main.version={{ .Version }}
- id: arm64-v9
binary: '{{ .ProjectName }}'
goos:
- darwin
- linux
goarch:
- arm64
goarm64:
- v9.0
env:
- CGO_ENABLED=0
flags:
- -trimpath
- -mod=readonly
ldflags: -s -w -X main.version={{ .Version }}
- id: amd64-v3
binary: '{{ .ProjectName }}'
goos:
- darwin
- freebsd
- linux
- netbsd
- openbsd
- windows
goarch:
- amd64
goamd64:
- v3
env:
- CGO_ENABLED=0
flags:
- -trimpath
- -mod=readonly
ldflags: -s -w -X main.version={{ .Version }}
archives:
- name_template: '{{ .ProjectName }}-{{ .Version }}-{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}'
- id: default
ids:
- default
- mips
name_template: '{{ .ProjectName }}-{{ .Version }}-{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}'
formats:
- tar.gz
wrap_in_directory: true
@@ -54,6 +115,26 @@ archives:
- LICENSE
- README.md
- SECURITY.md
- BEST_PRACTICES.md
- example.config.toml
- id: optimized
ids:
- arm64-v9
- amd64-v3
name_template: '{{ .ProjectName }}-{{ .Version }}-{{ .Os }}-{{ .Arch }}{{ if .Arm64 }}-{{ .Arm64 }}{{ end }}{{ if .Amd64 }}-{{ .Amd64 }}{{ end }}'
formats:
- tar.gz
wrap_in_directory: true
format_overrides:
- goos: windows
formats:
- zip
files:
- LICENSE
- README.md
- SECURITY.md
- BEST_PRACTICES.md
- example.config.toml
gomod:
proxy: true
+9 -3
View File
@@ -16,6 +16,12 @@ sources = ["**/*.go", "go.mod", "go.sum"]
outputs = ["mtg"]
run = "go build"
[tasks."build:prof"]
description = "Build binary with profiling enabled"
sources = ["**/*.go", "go.mod", "go.sum"]
outputs = ["mtg"]
run = "go build -tags prof"
[tasks.update]
description = "Update dependencies"
run = [
@@ -33,11 +39,11 @@ run = "govulncheck ./..."
[tasks.test]
description = "Run tests"
run = "go test -v ./..."
run = "go test -v -race ./..."
[tasks.covtest]
description = "Run tests with code coverage"
run = "go test -coverprofile=coverage.txt -covermode=atomic -parallel 2 -race -v ./..."
run = "go test -coverprofile=coverage.txt -covermode=atomic -count=2 -race -v ./..."
[tasks.test-all]
description = "Run all tests"
@@ -48,7 +54,7 @@ depends = [
[tasks."test:fuzz:client-hello"]
description = "Run fuzzy test for ClientHello"
run = "go test -v {{ vars.fuzzflags }} -fuzz=FuzzClientHello ./mtglib/internal/faketls"
run = "go test -v {{ vars.fuzzflags }} -fuzz=FuzzReadClientHello ./mtglib/internal/tls/fake"
[tasks."test:fuzz:client-handshake"]
description = "Run fuzzy test for ClientHandshake"
+55
View File
@@ -0,0 +1,55 @@
# Best practices
This is unfortunate, but since 2018 many things were changed. Most of them
became way worse. Previous iterations of censorship systems were very dumb,
DPI were primitive and filtered very obvious things. Nowadays they are
way more intelligent and it is very naive to treat them frivolously.
In 2026 is not enough to pretend that your mtg installation is a Microsoft
website that sits in Amsterdam Digital Ocean location. Now your installation
has to be a website that is mtg in disguise. Yes, it requires a bit more effort
but this effort is probably less than rotating proxies each other day.
mtproto traffic, even with FakeTLS, has its specifics that are probably
very well known by DPI systems. These specifics are not something unique but
could mark an IP address as suspicious. Now let's think:
1. You have a proxy in Amsterdam Digital Ocean that tells it is microsoft.com
how hard could it be to find out that this is probably fake? 1 or probably 2
DNS queries for `microsoft.com`? In case of some CDN, there are ECS-powered
resolvers that are very capable to return results from POV of some subnets.
If censor sees no relevant results, will they be afraid to block IP?
2. You have a proxy in Amsterdam Digital Ocean that tells it is a website from
the same public subnet. But not the same. Would it be hard to make these DNS
queries and ban IP?
The correct way of having this proxy is following:
1. Register a domain name
2. Get some VPS, probably in your domestic location
3. Set that domain name from a step 1 to IP address of that VPS
4. Generate a couple of HTML pages by LLMs or even copy them from elsewhere
5. Set some webserver and issue TLS certificates with Let's Encrypt or any other
name
6. Set mtg before this webserver.
7. Use sing-box or anything like that to provide local socks5 interface and
have VPNized uplinks
8. Set up mtg to use socks5 from a 7 step.
In that case you will get a match of DNS and SNI in requests. As a side effect,
your proxy will work with XTLS and its friends: XTLS in sniff mode ignores
IP address a client wants to connect to. Instead, it reads SNI and connect
to resolved address: a clever idea if user does not have a trustworthy DNS
set up.
Yes, this is much longer that usual technique, and requires more effort. But
this is could probably be very well automated to some reasonable extent.
Unfortunately, this is a best practice right now.
Do not also forget about other implementation, like
[telemt](https://github.com/telemt/telemt). Try everything. Use VPNs. It does
not really matter which project you are going to use as long it helps you to
stay connected.
_March 2026._
+26 -7
View File
@@ -5,17 +5,35 @@ FROM golang:1.26-alpine AS build
ENV CGO_ENABLED=0
# this is done for backward compatibility: before that we mounted a config
# into /config.toml. Some application allow mounting directories only,
# so it makes problems. So, instead we are going to do 2 steps:
# 1. Create /config/config.toml as a symlink to /config.toml
# 2. Force /mtg to use /config/config.toml
#
# it helps in both ways: users with directories could use /config directory
# and overlap a symlink by their bind mount. Old users could continue using
# /config.toml as a real config.
RUN set -x \
&& apk --no-cache --update add \
bash \
ca-certificates \
git
&& mkdir -p /config \
&& ln -sv /config.toml /config/config.toml
COPY . /app
RUN --mount=type=cache,target=/var/cache/apk \
set -x \
&& apk --update add \
bash \
ca-certificates \
git
COPY go.mod go.sum /app/
WORKDIR /app
RUN go mod download
COPY . /app
RUN set -x \
&& version="$(git describe --exact-match HEAD 2>/dev/null || git describe --tags --always)" \
&& version="$(git describe --exact-match HEAD 2>/dev/null || git describe --tags --always 2>/dev/null || echo dev)" \
&& go build \
-trimpath \
-mod=readonly \
@@ -30,8 +48,9 @@ RUN set -x \
FROM scratch
ENTRYPOINT ["/mtg"]
CMD ["run", "/config.toml"]
CMD ["run", "/config/config.toml"]
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=build /app/mtg /mtg
COPY --from=build /app/example.config.toml /config.toml
COPY --from=build /config /config
+116
View File
@@ -29,6 +29,8 @@ are the most notable:
* [Official](https://github.com/TelegramMessenger/MTProxy)
* [Python](https://github.com/alexbers/mtprotoproxy)
* [Erlang](https://github.com/seriyps/mtproto_proxy)
* [Teleproxy (C)](https://github.com/teleproxy/teleproxy)
* [mtproto.zig (Zig)](https://github.com/sleep3r/mtproto.zig)
* [Telemt (Rust)](https://github.com/telemt/telemt)
You can use any of these. They work great and all implementations have
@@ -38,6 +40,33 @@ goal: to give a possibility to connect to Telegram in a restricted,
censored environment. But it does it slightly differently in details
that probably matter.
* **Domain fronting**
For years mtg supports domain fronting. This technique means that it fallbacks
to accessing a real website in case if request fails. It could fail by many
reasons: anti-replay protection, accidental access to the webserver or
stale request. Anyway, if mtg rejects this request, it does not break a
connection. It connects to the websites and replicates everything that client
has sent, and simply proxies it back as is. Users will see a response from
the real website, _byte-to-byte identical_ to the response of the real netloc.
* **Doppelganger**
mtg also is a doppelganger of the website it fronts. Sure, with domain fronting
users will see replies of the real website in case if something will go wrong.
But what about such cases when _everything is fine_?
In that case mtg mimics TLS connection statistical characteristics as close as
possible. Different application have different statistics of their patterns.
Big CDN steadily pumping the data, small websites burst with short easily
compressiable chunks of traffic.
mtg artificially emulates those delays to be statistically indistinguishable
from the real website even if it covers connection of the very specific app.
It also follows 2 most common patterns of traffic chunking, so censors
will have to put more resources to find out that we have Telegram here
but not a hookah webshop served by nginx.
* **Resource-efficient**
It has to be resource-efficient. It does not mean that you will see
@@ -64,6 +93,9 @@ that probably matter.
software. I also believe that in the case of throwout proxies, this
the feature is a useless luxury.
This is very controversial topic. Please read [rationale (in russian)](https://github.com/9seconds/mtg/issues/376#issuecomment-4118726699)
and use [mtg-multi](https://github.com/dolonet/mtg-multi) fork if you are disagree with.
* **No adtag support**
Please read [Version 2](#version-2) chapter.
@@ -93,6 +125,8 @@ that probably matter.
software (written in Golang) with a minimum effort + you can replace
some parts with those you want.
Please also to read about [best practices](https://github.com/9seconds/mtg/blob/master/BEST_PRACTICES.md).
### Version 2
If you use version 1.x before, you are probably noticed some major
@@ -272,6 +306,38 @@ For example, you've bought a VPS from [Digital
Ocean](https://www.digitalocean.com/). Then it might be a good idea to
generate a secret for _digitalocean.com_ then.
### Check configuration
There is a special command for secret verification:
```
$ mtg doctor /path/to/my/config.toml
Deprecated options
✅ All good
Time skewness
✅ Time drift is -607.048µs, but tolerate-time-skewness is 5s
Validate native network connectivity
✅ DC 1
✅ DC 2
✅ DC 3
✅ DC 4
✅ DC 5
✅ DC 203
Validate network connectivity with proxy socks5://127.0.0.1:1080
✅ DC 1
✅ DC 2
✅ DC 3
✅ DC 4
✅ DC 5
✅ DC 203
Validate fronting domain connectivity
✅ xx.xx.xx.xx:yyy is reachable
Validate SNI-DNS match
✅ IP address xx.xx.xx.xx matches secret hostname <REDACTED>
```
It aims to find out possible inconsistencies and problems with your
configuration. It makes sense to run it before executing any relevant commands.
### Simple run mode
@@ -351,6 +417,7 @@ ExecStart=/usr/local/bin/mtg run /etc/mtg.toml
Restart=always
RestartSec=3
DynamicUser=true
LimitNOFILE=65536
AmbientCapabilities=CAP_NET_BIND_SERVICE
[Install]
@@ -398,6 +465,55 @@ or if you are using docker:
$ docker exec mtg-proxy /mtg access /config.toml
```
## Doppelganger
mtg can mimic real websites, please take a look at relevant section in example
config file.
mtg comes with some very good precollected statistics coming from
[ok.ru](https://ok.ru/). It does not mean that you have to cover yourself
by pretending that mtg is _ok.ru_. **Do not do that: ok.ru comes from very specific
ASNs, but not from VPS providers you are going to use.** What I want to say
is that defaults are very good enough to use as is because ok.ru for public
pages has a very generic profile of TLS packets delay.
But for better results it is recommended to teach mtg about the website you
will use as a domain front. In order to do that, you need to specify URLs
from this website. Just go to it, open WebDeveloper console and pick up
random URLs. For better results they have to be **from the same domain name
you are going to use as a disguise** but serve light and heavy content: pages,
images etc. Do not use many, 2-3 will probably work.
mtg will crawl these pages periodically, accumulating statistics and
using it as you go.
```toml
[defense.doppelganger]
urls = [
"https://lalala.com/index.html",
"https://lalala.com/contacts.html",
]
```
This is not very necessary. Keep in mind these rules:
1. If you are not sure what is this all about, do nothing. Defaults are good.
2. All URLs must be HTTPS
3. All URLs should be from the same domain name (but this is not a rule)
4. Do not use a lot of pages. Use _different_ pages. mtg will start using this
statistics when it will accumulate enough anyway.
5. These URLs should be directly accessible from mtg without proxies whatsoever
6. Do not create huge raids. mtg will repeatedly crawl in raids, making N repeats.
Do not use high N, you do not want to be noticeable.
7. It makes no sense to have small delay between raids. Usually webservers
do not update their TLS settings each hour.
8. If you have some specific knowledge if webserver is using
[TLS Dynamic Record Sizing](https://blog.cloudflare.com/optimizing-tls-over-tcp-to-reduce-latency/), you
can use a very specific setting. This are Cloudflare, Go standard webservers,
[caddy](https://caddyserver.com/) and [H2O](https://h2o.examp1e.net/). If so,
you can enable `drs` setting.
9. **If you are not sure, touch nothing!**
## Metrics
Out of the box, mtg works with
+1 -1
View File
@@ -12,7 +12,7 @@ type StableBloomFilterTestSuite struct {
}
func (suite *StableBloomFilterTestSuite) TestOp() {
filter := antireplay.NewStableBloomFilter(500, 0.001)
filter := antireplay.NewStableBloomFilter(100000, 0.001)
suite.False(filter.SeenBefore([]byte{1, 2, 3}))
suite.False(filter.SeenBefore([]byte{4, 5, 6}))
BIN
View File
Binary file not shown.
+30
View File
@@ -0,0 +1,30 @@
package essentials
// TelegramCoreAddresses are publicly known addresses of Telegram core network.
var TelegramCoreAddresses = map[int][]string{
1: {
"149.154.175.50:443",
"[2001:b28:f23d:f001::a]:443",
},
2: {
"149.154.167.51:443",
"95.161.76.100:443",
"[2001:67c:04e8:f002::a]:443",
},
3: {
"149.154.175.100:443",
"[2001:b28:f23d:f003::a]:443",
},
4: {
"149.154.167.91:443",
"[2001:67c:04e8:f004::a]:443",
},
5: {
"149.154.171.5:443",
"[2001:b28:f23f:f005::a]:443",
},
203: {
"91.105.192.100:443",
"[2a0a:f280:0203:000a:5000:0000:0000:0100]:443",
},
}
+1 -1
View File
@@ -38,7 +38,7 @@ func (e EventStream) Send(ctx context.Context, evt mtglib.Event) {
select {
case <-ctx.Done():
case <-e.ctx.Done():
case e.chans[int(chanNo)%len(e.chans)] <- evt:
case e.chans[chanNo%uint32(len(e.chans))] <- evt:
}
}
+80 -5
View File
@@ -48,6 +48,13 @@ concurrency = 8192
# Only ipv4 connectivity is used
prefer-ip = "prefer-ipv6"
# Public IP addresses of this server. Used by 'mtg access' to generate
# proxy links and by 'mtg doctor' to validate SNI-DNS match.
# If not set, mtg tries to detect them automatically via ifconfig.co.
# Set these if ifconfig.co is unreachable from your server.
# public-ipv4 = "1.2.3.4"
# public-ipv6 = "2001:db8::1"
# If this setting is set, then mtg will try to get proxy updates from Telegram
# Usually this is completely fine to have it disabled, because mtg has a list
# of some core proxies hardcoded.
@@ -197,14 +204,82 @@ proxies = [
# define a global timeout on establishing of network connections. idle
# means a timeout on pumping data between sockset when nothing is
# happening.
#
# please be noticed that handshakes have no timeouts intentionally. You can
# find a reasoning here:
# https://www.ndss-symposium.org/wp-content/uploads/2020/02/23087-paper.pdf
[network.timeout]
tcp = "5s"
http = "10s"
idle = "1m"
idle = "5m"
handshake = "10s"
# this defines a configuration for TCP keep alives. Default values are taken
# from Golang default behavior.
[network.keep-alive]
disabled = false
# idle means a time period after which we start sending TCP Keep Alive probes
idle = "15s"
# interval is a period between 2 consecutive probes
interval = "15s"
# if we miss that many probes, a connection will be considered as a dead one.
count = 9
# mtg has to mimic real websites. It does not mean domain fronting, it also
# means that traffic characteristics should be similar to real world traffic.
# websites and applications behave differently, their traffic patterns are also
# different. Applications do bursts of RPC-style messages (or JSON communication,
# does not really matter), while websites pump heavy content in HTTP2 streams
#
# It means that statistically there is a different between traffic shape:
# delays between packets are also different.
# In order to avoid censorship detection based on these patterns, there is a
# mtg subsystem called "Doppelganger" that aims to mimic website statistics
# as close as it could.
#
# Delays between TLS packets are not constant. There are many factors
# that come in play. Application should generate some response, it could
# send some headers first and stream content with chunked encoding. So
# some first packets could come as soon as possible, with some delays
# after first ones. Such phenomenon is described by different statistic
# distribution. There are 2 distribution that describe it: lognormal
# distribution and Weibul distribution. Lognormal is all about steady streams
# of heavy content like a video. Weibul is great about short bursts like
# user who requested a static page an a couple of images.
[defense.doppelganger]
# This is a list of URLs that would be crawled by mtg to approximate delay
# statistics. They MUST be HTTPS urls.
#
# You can come to the website and collect different URLs, with light and
# heavy content. We recommend to search for CDNs.
urls = [
# "https://st-ok.cdn-vk.ru/res/react/vendor/clsx-2.1.1-amd.js"
]
# A collection is done in raids. Each raid makes this number of requests to
# each URL in this list. Do not use a huge number, 10 is probably ok.
repeats-per-raid = 10
# This is a duration between each raid. It makes no sense to have a small number
# here as you would start to make a noticeable activity. Usually traffic patterns
# do not change a lot, so do not expect different results if you request
# each 10 minutes.
raid-each = "6h"
# This enables dynamic tls record sizing.
#
# Some modern stacks and platforms start to use the technique that is called
# DRS. They start with small TLS packets and ramp up eventually. First packets
# are usually about MTU size, after that we get 4k and eventually max size.
# This is done with a good intention: to minimize a time to the first byte,
# so application could start doing something with the data right after first
# RTT.
#
# Apparently, about 90% of application do not employ this technique, they use
# max size always: nginx, apache, java stuff. But Golang tools, angie and
# some specific patches activate this technique.
#
# In order to mimic a real website we need to know something about software
# it uses. Usually nobody cares: openssl does 16384, Python does it, nginx
# does it. So this setting is disabled by default.
#
# https://blog.cloudflare.com/optimizing-tls-over-tcp-to-reduce-latency/
# https://aws.github.io/s2n-tls/usage-guide/ch08-record-sizes.html
# https://github.com/cloudflare/sslconfig/blob/master/patches/nginx__dynamic_tls_records.patch
drs = false
# Some countries do active probing on Telegram connections. This technique
# allows to protect from such effort.
+12 -11
View File
@@ -4,31 +4,32 @@ go 1.26
require (
github.com/OneOfOne/xxhash v1.2.8
github.com/alecthomas/kong v1.14.0
github.com/alecthomas/kong v1.15.0
github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5
github.com/babolivier/go-doh-client v0.0.0-20201028162107-a76cff4cb8b6
github.com/d4l3k/messagediff v1.2.1 // indirect
github.com/jarcoal/httpmock v1.0.8
github.com/mccutchen/go-httpbin v1.1.1
github.com/panjf2000/ants/v2 v2.11.5
github.com/panjf2000/ants/v2 v2.12.0
github.com/prometheus/client_golang v1.23.2
github.com/prometheus/common v0.67.5 // indirect
github.com/prometheus/procfs v0.20.0 // indirect
github.com/rs/zerolog v1.34.0
github.com/prometheus/procfs v0.20.1 // indirect
github.com/rs/zerolog v1.35.0
github.com/smira/go-statsd v1.3.4
github.com/stretchr/objx v0.5.2 // indirect
github.com/stretchr/testify v1.11.1
github.com/tylertreat/BoomFilters v0.0.0-20251117164519-53813c36cc1b
golang.org/x/crypto v0.48.0
golang.org/x/net v0.51.0
golang.org/x/sys v0.41.0
golang.org/x/crypto v0.49.0
golang.org/x/net v0.52.0
golang.org/x/sys v0.42.0
google.golang.org/protobuf v1.36.11 // indirect
)
require (
github.com/ncruces/go-dns v1.3.2
github.com/pelletier/go-toml/v2 v2.2.4
github.com/beevik/ntp v1.5.0
github.com/ncruces/go-dns v1.3.3
github.com/pelletier/go-toml/v2 v2.3.0
github.com/pires/go-proxyproto v0.11.0
github.com/things-go/go-socks5 v0.1.0
github.com/txthinking/socks5 v0.0.0-20251011041537-5c31f201a10e
@@ -49,8 +50,8 @@ require (
github.com/prometheus/client_model v0.6.2 // indirect
github.com/rogpeppe/go-internal v1.14.1 // indirect
github.com/txthinking/runnergroup v0.0.0-20250224021307-5864ffeb65ae // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
golang.org/x/sync v0.19.0 // indirect
go.yaml.in/yaml/v2 v2.4.4 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/tools v0.41.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
+24 -31
View File
@@ -2,8 +2,8 @@ github.com/OneOfOne/xxhash v1.2.8 h1:31czK/TI9sNkxIKfaUfGlU47BAxQ0ztGgd9vPyqimf8
github.com/OneOfOne/xxhash v1.2.8/go.mod h1:eZbhyaAYD41SGSSsnmcpxVoRiQ/MPUTjUdIIOT9Um7Q=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/kong v1.14.0 h1:gFgEUZWu2ZmZ+UhyZ1bDhuutbKN1nTtJTwh19Wsn21s=
github.com/alecthomas/kong v1.14.0/go.mod h1:wrlbXem1CWqUV5Vbmss5ISYhsVPkBb1Yo7YKJghju2I=
github.com/alecthomas/kong v1.15.0 h1:BVJstKbpO73zKpmIu+m/aLRrNmWwxXPIGTNin9VmLVI=
github.com/alecthomas/kong v1.15.0/go.mod h1:wrlbXem1CWqUV5Vbmss5ISYhsVPkBb1Yo7YKJghju2I=
github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b h1:mimo19zliBX/vSQ6PWWSL9lK8qwHozUj03+zLoEB8O0=
@@ -12,18 +12,18 @@ github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPd
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
github.com/babolivier/go-doh-client v0.0.0-20201028162107-a76cff4cb8b6 h1:4NNbNM2Iq/k57qEu7WfL67UrbPq1uFWxW4qODCohi+0=
github.com/babolivier/go-doh-client v0.0.0-20201028162107-a76cff4cb8b6/go.mod h1:J29hk+f9lJrblVIfiJOtTFk+OblBawmib4uz/VdKzlg=
github.com/beevik/ntp v1.5.0 h1:y+uj/JjNwlY2JahivxYvtmv4ehfi3h74fAuABB9ZSM4=
github.com/beevik/ntp v1.5.0/go.mod h1:mJEhBrwT76w9D+IfOEGvuzyuudiW9E52U2BaTrMOYow=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/d4l3k/messagediff v1.2.1 h1:ZcAIMYsUg0EAp9X+tt8/enBE/Q8Yd5kzPynLyKptt9U=
github.com/d4l3k/messagediff v1.2.1/go.mod h1:Oozbb1TVXFac9FtSIxHBMnBCq2qeH/2KkEQxENCrlLo=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
@@ -38,11 +38,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mccutchen/go-httpbin v1.1.1 h1:aEws49HEJEyXHLDnshQVswfUlCVoS8g6h9YaDyaW7RE=
@@ -51,17 +48,16 @@ github.com/miekg/dns v1.1.51 h1:0+Xg7vObnhrz/4ZCZcZh7zPXlmU0aveS2HDBd0m0qSo=
github.com/miekg/dns v1.1.51/go.mod h1:2Z9d3CP1LQWihRZUf29mQ19yDThaI4DAYzte2CaQW5c=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/ncruces/go-dns v1.3.2 h1:kBLuUZBgkQ4qF4WDXZRQ4rG0Gk6sLVJQ5tESkWrxUa0=
github.com/ncruces/go-dns v1.3.2/go.mod h1:tuzixNY8PY/M7yUzcvRbUaeLs3ifIdydpi5H2bfRU+s=
github.com/panjf2000/ants/v2 v2.11.5 h1:a7LMnMEeux/ebqTux140tRiaqcFTV0q2bEHF03nl6Rg=
github.com/panjf2000/ants/v2 v2.11.5/go.mod h1:8u92CYMUc6gyvTIw8Ru7Mt7+/ESnJahz5EVtqfrilek=
github.com/ncruces/go-dns v1.3.3 h1:59OV7XoJrTCoUMZjWRVs4GOjtntMTZqiQ5Mn+BT13hk=
github.com/ncruces/go-dns v1.3.3/go.mod h1:tuzixNY8PY/M7yUzcvRbUaeLs3ifIdydpi5H2bfRU+s=
github.com/panjf2000/ants/v2 v2.12.0 h1:u9JhESo83i/GkZnhfTNuFMMWcNt7mnV1bGJ6FT4wXH8=
github.com/panjf2000/ants/v2 v2.12.0/go.mod h1:tSQuaNQ6r6NRhPt+IZVUevvDyFMTs+eS4ztZc52uJTY=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pelletier/go-toml/v2 v2.3.0 h1:k59bC/lIZREW0/iVaQR8nDHxVq8OVlIzYCOJf421CaM=
github.com/pelletier/go-toml/v2 v2.3.0/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pires/go-proxyproto v0.11.0 h1:gUQpS85X/VJMdUsYyEgyn59uLJvGqPhJV5YvG68wXH4=
github.com/pires/go-proxyproto v0.11.0/go.mod h1:ZKAAyp3cgy5Y5Mo4n9AlScrkCZwUy0g3Jf+slqQVcuU=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
@@ -70,13 +66,12 @@ github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNw
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4=
github.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw=
github.com/prometheus/procfs v0.20.0 h1:AA7aCvjxwAquZAlonN7888f2u4IN8WVeFgBi4k82M4Q=
github.com/prometheus/procfs v0.20.0/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=
github.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEycfc=
github.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
github.com/rs/zerolog v1.34.0 h1:k43nTLIwcTVQAncfCw4KZ2VY6ukYoZaBPNOE8txlOeY=
github.com/rs/zerolog v1.34.0/go.mod h1:bJsvje4Z08ROH4Nhs5iH600c3IkWhwp44iRc54W6wYQ=
github.com/rs/zerolog v1.35.0 h1:VD0ykx7HMiMJytqINBsKcbLS+BJ4WYjz+05us+LRTdI=
github.com/rs/zerolog v1.35.0/go.mod h1:EjML9kdfa/RMA7h/6z6pYmq1ykOuA8/mjWaEvGI+jcw=
github.com/smira/go-statsd v1.3.4 h1:kBYWcLSGT+qC6JVbvfz48kX7mQys32fjDOPrfmsSx2c=
github.com/smira/go-statsd v1.3.4/go.mod h1:RjdsESPgDODtg1VpVVf9MJrEW2Hw0wtRNbmB1CAhu6A=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -105,12 +100,12 @@ github.com/yl2chen/cidranger v1.0.2/go.mod h1:9U1yz7WPYDwf0vpNWFaeRh0bjwz5RVgRy/
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts=
golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos=
golang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4=
golang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.7.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.32.0 h1:9F4d3PHLljb6x//jOyokMv3eX+YDeepZSEo3mFJy93c=
@@ -119,24 +114,22 @@ golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLL
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.51.0 h1:94R/GTO7mt3/4wIKpcR5gkGmRLOuE/2hNGeWq/GBIFo=
golang.org/x/net v0.51.0/go.mod h1:aamm+2QF5ogm02fjy5Bb7CQ0WMt1/WVM7FtyaTLlA9Y=
golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
+8 -42
View File
@@ -1,22 +1,16 @@
package cli
import (
"context"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"sync"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/internal/config"
"github.com/9seconds/mtg/v2/internal/utils"
"github.com/9seconds/mtg/v2/mtglib"
)
type accessResponse struct {
@@ -65,7 +59,10 @@ func (a *Access) Run(cli *CLI, version string) error {
wg.Go(func() {
ip := a.PublicIPv4
if ip == nil {
ip = a.getIP(ntw, "tcp4")
ip = conf.PublicIPv4.Get(nil)
}
if ip == nil {
ip = getIP(ntw, "tcp4")
}
if ip != nil {
@@ -77,7 +74,10 @@ func (a *Access) Run(cli *CLI, version string) error {
wg.Go(func() {
ip := a.PublicIPv6
if ip == nil {
ip = a.getIP(ntw, "tcp6")
ip = conf.PublicIPv6.Get(nil)
}
if ip == nil {
ip = getIP(ntw, "tcp6")
}
if ip != nil {
@@ -100,40 +100,6 @@ func (a *Access) Run(cli *CLI, version string) error {
return nil
}
func (a *Access) getIP(ntw mtglib.Network, protocol string) net.IP {
client := ntw.MakeHTTPClient(func(ctx context.Context, network, address string) (essentials.Conn, error) {
return ntw.DialContext(ctx, protocol, address) //nolint: wrapcheck
})
req, err := http.NewRequest(http.MethodGet, "https://ifconfig.co", nil) //nolint: noctx
if err != nil {
panic(err)
}
req.Header.Add("Accept", "text/plain")
resp, err := client.Do(req)
if err != nil {
return nil
}
if resp.StatusCode != http.StatusOK {
return nil
}
defer func() {
io.Copy(io.Discard, resp.Body) //nolint: errcheck
resp.Body.Close() //nolint: errcheck
}()
data, err := io.ReadAll(resp.Body)
if err != nil {
return nil
}
return net.ParseIP(strings.TrimSpace(string(data)))
}
func (a *Access) makeURLs(conf *config.Config, ip net.IP) *accessResponseURLs {
if ip == nil {
return nil
+1
View File
@@ -4,6 +4,7 @@ import "github.com/alecthomas/kong"
type CLI struct {
GenerateSecret GenerateSecret `kong:"cmd,help='Generate new proxy secret'"`
Doctor Doctor `kong:"cmd,help='Check that proxy can run correctly'"`
Access Access `kong:"cmd,help='Print access information.'"`
Run Run `kong:"cmd,help='Run proxy.'"`
SimpleRun SimpleRun `kong:"cmd,help='Run proxy without config file.'"`
+381
View File
@@ -0,0 +1,381 @@
package cli
import (
"context"
"errors"
"fmt"
"maps"
"net"
"os"
"slices"
"strconv"
"strings"
"text/template"
"time"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/internal/config"
"github.com/9seconds/mtg/v2/internal/utils"
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/network/v2"
"github.com/beevik/ntp"
)
var (
tplError = template.Must(
template.New("").Parse(" ‼️ {{ .description }}: {{ .error }}\n"),
)
tplWDeprecatedConfig = template.Must(
template.New("").
Parse(` ⚠️ Option {{ .old | printf "%q" }}{{ if .old_section }} from section [{{ .old_section }}]{{ end }} is deprecated and will be removed in v{{ .when }}. Please use {{ .new | printf "%q" }}{{ if .new_section }} in [{{ .new_section }}] section{{ end }} instead.` + "\n"),
)
tplOTimeSkewness = template.Must(
template.New("").
Parse(" ✅ Time drift is {{ .drift }}, but tolerate-time-skewness is {{ .value }}\n"),
)
tplWTimeSkewness = template.Must(
template.New("").
Parse(" ⚠️ Time drift is {{ .drift }}, but tolerate-time-skewness is {{ .value }}. Please check ntp.\n"),
)
tplETimeSkewness = template.Must(
template.New("").
Parse(" ❌ Time drift is {{ .drift }}, but tolerate-time-skewness is {{ .value }}. You will get many rejected connections!\n"),
)
tplODCConnect = template.Must(
template.New("").Parse(" ✅ DC {{ .dc }}\n"),
)
tplEDCConnect = template.Must(
template.New("").Parse(" ❌ DC {{ .dc }}: {{ .error }}\n"),
)
tplODNSSNIMatch = template.Must(
template.New("").Parse(" ✅ IP address {{ .ip }} matches secret hostname {{ .hostname }}\n"),
)
tplEDNSSNIMatch = template.Must(
template.New("").Parse(" ❌ Hostname {{ .hostname }} {{ if .resolved }}is resolved to {{ .resolved }} addresses, not {{ if .ip4 }}{{ .ip4 }}{{ else }}{{ .ip6 }}{{ end }}{{ else }}cannot be resolved to any host{{ end }}\n"),
)
tplOFrontingDomain = template.Must(
template.New("").Parse(" ✅ {{ .address }} is reachable\n"),
)
tplEFrontingDomain = template.Must(
template.New("").Parse(" ❌ {{ .address }}: {{ .error }}\n"),
)
)
type Doctor struct {
conf *config.Config
ConfigPath string `kong:"arg,required,type='existingfile',help='Path to the configuration file.',name='config-path'"` //nolint: lll
}
func (d *Doctor) Run(cli *CLI, version string) error {
conf, err := utils.ReadConfig(d.ConfigPath)
if err != nil {
return fmt.Errorf("cannot init config: %w", err)
}
d.conf = conf
fmt.Println("Deprecated options")
everythingOK := d.checkDeprecatedConfig()
fmt.Println("Time skewness")
everythingOK = d.checkTimeSkewness() && everythingOK
resolver, err := network.GetDNS(conf.GetDNS())
if err != nil {
return fmt.Errorf("cannot create DNS resolver: %w", err)
}
base := network.New(
resolver,
"",
conf.Network.Timeout.TCP.Get(10*time.Second),
conf.Network.Timeout.HTTP.Get(0),
conf.Network.Timeout.Idle.Get(0),
net.KeepAliveConfig{
Enable: !conf.Network.KeepAlive.Disabled.Get(false),
Idle: conf.Network.KeepAlive.Idle.Get(0),
Interval: conf.Network.KeepAlive.Interval.Get(0),
Count: int(conf.Network.KeepAlive.Count.Get(0)),
},
)
fmt.Println("Validate native network connectivity")
everythingOK = d.checkNetwork(base) && everythingOK
for _, url := range conf.Network.Proxies {
value, err := network.NewProxyNetwork(base, url.Get(nil))
if err != nil {
return err
}
fmt.Printf("Validate network connectivity with proxy %s\n", url.Get(nil))
everythingOK = d.checkNetwork(value) && everythingOK
}
fmt.Println("Validate fronting domain connectivity")
everythingOK = d.checkFrontingDomain(base) && everythingOK
fmt.Println("Validate SNI-DNS match")
everythingOK = d.checkSecretHost(resolver, base) && everythingOK
if !everythingOK {
os.Exit(1)
}
return nil
}
func (d *Doctor) checkDeprecatedConfig() bool {
ok := true
if d.conf.DomainFrontingIP.Value != nil {
ok = false
tplWDeprecatedConfig.Execute(os.Stdout, map[string]string{ //nolint: errcheck
"when": "2.3.0",
"old": "domain-fronting-ip",
"old_section": "",
"new": "ip",
"new_section": "domain-fronting",
})
}
if d.conf.DomainFrontingPort.Value != 0 {
ok = false
tplWDeprecatedConfig.Execute(os.Stdout, map[string]string{ //nolint: errcheck
"when": "2.3.0",
"old": "domain-fronting-port",
"old_section": "",
"new": "port",
"new_section": "domain-fronting",
})
}
if d.conf.DomainFrontingProxyProtocol.Value {
ok = false
tplWDeprecatedConfig.Execute(os.Stdout, map[string]string{ //nolint: errcheck
"when": "2.3.0",
"old": "domain-fronting-proxy-protocol",
"old_section": "",
"new": "proxy-protocol",
"new_section": "domain-fronting",
})
}
if d.conf.Network.DOHIP.Value != nil {
ok = false
tplWDeprecatedConfig.Execute(os.Stdout, map[string]string{ //nolint: errcheck
"when": "2.3.0",
"old": "doh-ip",
"old_section": "network",
"new": "dns",
"new_section": "network",
})
}
if ok {
fmt.Println(" ✅ All good")
}
return ok
}
func (d *Doctor) checkTimeSkewness() bool {
response, err := ntp.Query("0.pool.ntp.org")
if err != nil {
tplError.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"description": "cannot access ntp pool",
"error": err,
})
return false
}
skewness := response.ClockOffset.Abs()
confValue := d.conf.TolerateTimeSkewness.Get(mtglib.DefaultTolerateTimeSkewness)
diff := float64(skewness) / float64(confValue)
tplData := map[string]any{
"drift": response.ClockOffset,
"value": confValue,
}
switch {
case diff < 0.3:
tplOTimeSkewness.Execute(os.Stdout, tplData) //nolint: errcheck
return true
case diff < 0.7:
tplWTimeSkewness.Execute(os.Stdout, tplData) //nolint: errcheck
default:
tplETimeSkewness.Execute(os.Stdout, tplData) //nolint: errcheck
}
return false
}
func (d *Doctor) checkNetwork(ntw mtglib.Network) bool {
dcs := slices.Collect(maps.Keys(essentials.TelegramCoreAddresses))
slices.Sort(dcs)
ok := true
for _, dc := range dcs {
err := d.checkNetworkAddresses(ntw, essentials.TelegramCoreAddresses[dc])
if err == nil {
tplODCConnect.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"dc": dc,
})
} else {
tplEDCConnect.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"dc": dc,
"error": err,
})
ok = false
}
}
return ok
}
func (d *Doctor) checkNetworkAddresses(ntw mtglib.Network, addresses []string) error {
checkAddresses := []string{}
switch d.conf.PreferIP.Get("prefer-ip4") {
case "only-ipv4":
for _, addr := range addresses {
host, _, err := net.SplitHostPort(addr)
if err != nil {
panic(err)
}
if ip := net.ParseIP(host); ip != nil && ip.To4() != nil {
checkAddresses = append(checkAddresses, addr)
}
}
case "only-ipv6":
for _, addr := range addresses {
host, _, err := net.SplitHostPort(addr)
if err != nil {
panic(err)
}
if ip := net.ParseIP(host); ip != nil && ip.To4() == nil {
checkAddresses = append(checkAddresses, addr)
}
}
default:
checkAddresses = addresses
}
if len(checkAddresses) == 0 {
return fmt.Errorf("no suitable addresses after IP version filtering")
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
var (
conn net.Conn
err error
)
for _, addr := range checkAddresses {
conn, err = ntw.DialContext(ctx, "tcp", addr)
if err != nil {
continue
}
conn.Close() //nolint: errcheck
return nil
}
return err
}
func (d *Doctor) checkFrontingDomain(ntw mtglib.Network) bool {
host := d.conf.Secret.Host
if ip := d.conf.GetDomainFrontingIP(nil); ip != "" {
host = ip
}
port := d.conf.GetDomainFrontingPort(mtglib.DefaultDomainFrontingPort)
address := net.JoinHostPort(host, strconv.Itoa(int(port)))
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
dialer := ntw.NativeDialer()
conn, err := dialer.DialContext(ctx, "tcp", address)
if err != nil {
tplEFrontingDomain.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"address": address,
"error": err,
})
return false
}
conn.Close() //nolint: errcheck
tplOFrontingDomain.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"address": address,
})
return true
}
func (d *Doctor) checkSecretHost(resolver *net.Resolver, ntw mtglib.Network) bool {
addresses, err := resolver.LookupIPAddr(context.Background(), d.conf.Secret.Host)
if err != nil {
tplError.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"description": fmt.Sprintf("cannot resolve DNS name of %s", d.conf.Secret.Host),
"error": err,
})
return false
}
ourIP4 := d.conf.PublicIPv4.Get(nil)
if ourIP4 == nil {
ourIP4 = getIP(ntw, "tcp4")
}
ourIP6 := d.conf.PublicIPv6.Get(nil)
if ourIP6 == nil {
ourIP6 = getIP(ntw, "tcp6")
}
if ourIP4 == nil && ourIP6 == nil {
tplError.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"description": "cannot detect public IP address",
"error": errors.New("cannot detect automatically and public-ipv4/public-ipv6 are not set in config"),
})
return false
}
strAddresses := []string{}
for _, value := range addresses {
if (ourIP4 != nil && value.IP.String() == ourIP4.String()) ||
(ourIP6 != nil && value.IP.String() == ourIP6.String()) {
tplODNSSNIMatch.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"ip": value.IP,
"hostname": d.conf.Secret.Host,
})
return true
}
strAddresses = append(strAddresses, `"`+value.IP.String()+`"`)
}
tplEDNSSNIMatch.Execute(os.Stdout, map[string]any{ //nolint: errcheck
"hostname": d.conf.Secret.Host,
"resolved": strings.Join(strAddresses, ", "),
"ip4": ourIP4,
"ip6": ourIP6,
})
return false
}
+21 -2
View File
@@ -46,13 +46,19 @@ func makeNetwork(conf *config.Config, version string) (mtglib.Network, error) {
base := network.New(
resolver,
"mtg/"+version,
"",
conf.Network.Timeout.TCP.Get(0),
conf.Network.Timeout.HTTP.Get(0),
conf.Network.Timeout.Idle.Get(0),
net.KeepAliveConfig{
Enable: !conf.Network.KeepAlive.Disabled.Get(false),
Idle: conf.Network.KeepAlive.Idle.Get(0),
Interval: conf.Network.KeepAlive.Interval.Get(0),
Count: int(conf.Network.KeepAlive.Count.Get(0)),
},
)
proxyDialers := make([]network.Network, len(conf.Network.Proxies))
proxyDialers := make([]mtglib.Network, len(conf.Network.Proxies))
for idx, v := range conf.Network.Proxies {
value, err := network.NewProxyNetwork(base, v.Get(nil))
if err != nil {
@@ -239,6 +245,11 @@ func runProxy(conf *config.Config, version string) error { //nolint: funlen
return fmt.Errorf("cannot build ip allowlist: %w", err)
}
doppelGangerURLs := make([]string, len(conf.Defense.Doppelganger.URLs))
for i, v := range conf.Defense.Doppelganger.URLs {
doppelGangerURLs[i] = v.String()
}
opts := mtglib.ProxyOpts{
Logger: logger,
Network: ntw,
@@ -248,6 +259,7 @@ func runProxy(conf *config.Config, version string) error { //nolint: funlen
EventStream: eventStream,
Secret: conf.Secret,
Concurrency: conf.GetConcurrency(mtglib.DefaultConcurrency),
DomainFrontingPort: conf.GetDomainFrontingPort(mtglib.DefaultDomainFrontingPort),
DomainFrontingIP: conf.GetDomainFrontingIP(nil),
DomainFrontingProxyProtocol: conf.GetDomainFrontingProxyProtocol(false),
@@ -256,6 +268,13 @@ func runProxy(conf *config.Config, version string) error { //nolint: funlen
AllowFallbackOnUnknownDC: conf.AllowFallbackOnUnknownDC.Get(false),
TolerateTimeSkewness: conf.TolerateTimeSkewness.Value,
IdleTimeout: conf.Network.Timeout.Idle.Get(mtglib.DefaultIdleTimeout),
HandshakeTimeout: conf.Network.Timeout.Handshake.Get(mtglib.DefaultHandshakeTimeout),
DoppelGangerURLs: doppelGangerURLs,
DoppelGangerPerRaid: conf.Defense.Doppelganger.Repeats.Get(mtglib.DoppelGangerPerRaid),
DoppelGangerEach: conf.Defense.Doppelganger.UpdateEach.Get(mtglib.DoppelGangerEach),
DoppelGangerDRS: conf.Defense.Doppelganger.DRS.Get(false),
}
proxy, err := mtglib.NewProxy(opts)
+51
View File
@@ -0,0 +1,51 @@
package cli
import (
"context"
"io"
"net"
"net/http"
"strings"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/mtglib"
)
func getIP(ntw mtglib.Network, protocol string) net.IP {
dialer := ntw.NativeDialer()
client := ntw.MakeHTTPClient(func(ctx context.Context, network, address string) (essentials.Conn, error) {
conn, err := dialer.DialContext(ctx, protocol, address)
if err != nil {
return nil, err
}
return essentials.WrapNetConn(conn), err
})
req, err := http.NewRequest(http.MethodGet, "https://ifconfig.co", nil) //nolint: noctx
if err != nil {
panic(err)
}
req.Header.Add("Accept", "text/plain")
resp, err := client.Do(req)
if err != nil {
return nil
}
if resp.StatusCode != http.StatusOK {
return nil
}
defer func() {
io.Copy(io.Discard, resp.Body) //nolint: errcheck
resp.Body.Close() //nolint: errcheck
}()
data, err := io.ReadAll(resp.Body)
if err != nil {
return nil
}
return net.ParseIP(strings.TrimSpace(string(data)))
}
+27 -5
View File
@@ -35,6 +35,8 @@ type Config struct {
DomainFrontingProxyProtocol TypeBool `json:"domainFrontingProxyProtocol"`
TolerateTimeSkewness TypeDuration `json:"tolerateTimeSkewness"`
Concurrency TypeConcurrency `json:"concurrency"`
PublicIPv4 TypeIP `json:"publicIpv4"`
PublicIPv6 TypeIP `json:"publicIpv6"`
DomainFronting struct {
IP TypeIP `json:"ip"`
Port TypePort `json:"port"`
@@ -47,15 +49,28 @@ type Config struct {
MaxSize TypeBytes `json:"maxSize"`
ErrorRate TypeErrorRate `json:"errorRate"`
} `json:"antiReplay"`
Blocklist ListConfig `json:"blocklist"`
Allowlist ListConfig `json:"allowlist"`
Blocklist ListConfig `json:"blocklist"`
Allowlist ListConfig `json:"allowlist"`
Doppelganger struct {
URLs []TypeHttpsURL `json:"urls"`
Repeats TypeConcurrency `json:"repeats_per_raid"`
UpdateEach TypeDuration `json:"raid_each"`
DRS TypeBool `json:"drs"`
} `json:"doppelganger"`
} `json:"defense"`
Network struct {
Timeout struct {
TCP TypeDuration `json:"tcp"`
HTTP TypeDuration `json:"http"`
Idle TypeDuration `json:"idle"`
TCP TypeDuration `json:"tcp"`
HTTP TypeDuration `json:"http"`
Idle TypeDuration `json:"idle"`
Handshake TypeDuration `json:"handshake"`
} `json:"timeout"`
KeepAlive struct {
Disabled TypeBool `json:"disabled"`
Idle TypeDuration `json:"idle"`
Interval TypeDuration `json:"interval"`
Count TypeConcurrency `json:"count"`
} `json:"keepAlive"`
DOHIP TypeIP `json:"dohIp"`
DNS TypeDNSURI `json:"dns"`
Proxies []TypeProxyURL `json:"proxies"`
@@ -78,6 +93,13 @@ type Config struct {
} `json:"stats"`
}
func (c *Config) GetConcurrency(defaultValue uint) uint {
if concurrency := c.Concurrency.Get(0); concurrency != 0 {
return concurrency
}
return c.Concurrency.Get(defaultValue)
}
func (c *Config) GetDNS() *url.URL {
var dohURL *url.URL
+26
View File
@@ -42,6 +42,32 @@ func (suite *ConfigTestSuite) TestParseMinimalConfig() {
suite.Equal("0.0.0.0:3128", conf.BindTo.String())
}
func (suite *ConfigTestSuite) TestParsePublicIP() {
conf, err := config.Parse(suite.ReadConfig("public_ip.toml"))
suite.NoError(err)
suite.Equal("203.0.113.1", conf.PublicIPv4.Get(nil).String())
suite.Equal("2001:db8::1", conf.PublicIPv6.Get(nil).String())
}
func (suite *ConfigTestSuite) TestParsePublicIPv4Only() {
conf, err := config.Parse(suite.ReadConfig("public_ip_v4_only.toml"))
suite.NoError(err)
suite.Equal("203.0.113.1", conf.PublicIPv4.Get(nil).String())
suite.Nil(conf.PublicIPv6.Get(nil))
}
func (suite *ConfigTestSuite) TestParsePublicIPInvalid() {
_, err := config.Parse(suite.ReadConfig("public_ip_invalid.toml"))
suite.Error(err)
}
func (suite *ConfigTestSuite) TestParsePublicIPNotSet() {
conf, err := config.Parse(suite.ReadConfig("minimal.toml"))
suite.NoError(err)
suite.Nil(conf.PublicIPv4.Get(nil))
suite.Nil(conf.PublicIPv6.Get(nil))
}
func (suite *ConfigTestSuite) TestString() {
conf, err := config.Parse(suite.ReadConfig("minimal.toml"))
suite.NoError(err)
+18 -3
View File
@@ -21,6 +21,8 @@ type tomlConfig struct {
DomainFrontingProxyProtocol bool `toml:"domain-fronting-proxy-protocol" json:"domainFrontingProxyProtocol,omitempty"`
TolerateTimeSkewness string `toml:"tolerate-time-skewness" json:"tolerateTimeSkewness,omitempty"`
Concurrency uint `toml:"concurrency" json:"concurrency,omitempty"`
PublicIPv4 string `toml:"public-ipv4" json:"publicIpv4,omitempty"`
PublicIPv6 string `toml:"public-ipv6" json:"publicIpv6,omitempty"`
DomainFronting struct {
IP string `toml:"ip" json:"ip,omitempty"`
Port uint `toml:"port" json:"port,omitempty"`
@@ -44,13 +46,26 @@ type tomlConfig struct {
URLs []string `toml:"urls" json:"urls,omitempty"`
UpdateEach string `toml:"update-each" json:"updateEach,omitempty"`
} `toml:"allowlist" json:"allowlist,omitempty"`
Doppelganger struct {
URLs []string `toml:"urls" json:"urls,omitempty"`
Repeats uint `toml:"repeats-per-raid" json:"repeats_per_raid,omitempty"`
UpdateEach string `toml:"raid-each" json:"raid_each,omitempty"`
DRS bool `toml:"drs" json:"drs,omitempty"`
} `toml:"doppelganger" json:"doppelganger,omitempty"`
} `toml:"defense" json:"defense,omitempty"`
Network struct {
Timeout struct {
TCP string `toml:"tcp" json:"tcp,omitempty"`
HTTP string `toml:"http" json:"http,omitempty"`
Idle string `toml:"idle" json:"idle,omitempty"`
TCP string `toml:"tcp" json:"tcp,omitempty"`
HTTP string `toml:"http" json:"http,omitempty"`
Idle string `toml:"idle" json:"idle,omitempty"`
Handshake string `toml:"handshake" json:"handshake,omitempty"`
} `toml:"timeout" json:"timeout,omitempty"`
KeepAlive struct {
Disabled bool `toml:"disabled" json:"disabled,omitempty"`
Idle string `toml:"idle" json:"idle,omitempty"`
Interval string `toml:"interval" json:"interval,omitempty"`
Count uint `toml:"count" json:"count,omitempty"`
} `toml:"keep-alive" json:"keepAlive,omitempty"`
DOHIP string `toml:"doh-ip" json:"dohIp,omitempty"`
DNS string `toml:"dns" json:"dns,omitempty"`
Proxies []string `toml:"proxies" json:"proxies,omitempty"`
+4
View File
@@ -0,0 +1,4 @@
secret = "7oe1GqLy6TBc38CV3jx7q09nb29nbGUuY29t"
bind-to = "0.0.0.0:3128"
public-ipv4 = "203.0.113.1"
public-ipv6 = "2001:db8::1"
+3
View File
@@ -0,0 +1,3 @@
secret = "7oe1GqLy6TBc38CV3jx7q09nb29nbGUuY29t"
bind-to = "0.0.0.0:3128"
public-ipv4 = "not-an-ip"
+3
View File
@@ -0,0 +1,3 @@
secret = "7oe1GqLy6TBc38CV3jx7q09nb29nbGUuY29t"
bind-to = "0.0.0.0:3128"
public-ipv4 = "203.0.113.1"
+53
View File
@@ -0,0 +1,53 @@
package config
import (
"fmt"
"net/url"
)
type TypeHttpsURL struct {
Value *url.URL
}
func (t *TypeHttpsURL) Set(value string) error {
parsedURL, err := url.Parse(value)
if err != nil {
return fmt.Errorf("value is not correct URL (%s): %w", value, err)
}
if parsedURL.Host == "" {
return fmt.Errorf("url has to have a schema: %s", value)
}
if parsedURL.Scheme != "https" {
return fmt.Errorf("unsupported schema: %s", parsedURL.Scheme)
}
t.Value = parsedURL
return nil
}
func (t *TypeHttpsURL) Get(defaultValue *url.URL) *url.URL {
if t.Value == nil {
return defaultValue
}
return t.Value
}
func (t *TypeHttpsURL) UnmarshalText(data []byte) error {
return t.Set(string(data))
}
func (t TypeHttpsURL) MarshalText() ([]byte, error) {
return []byte(t.String()), nil
}
func (t TypeHttpsURL) String() string {
if t.Value == nil {
return ""
}
return t.Value.String()
}
+100
View File
@@ -0,0 +1,100 @@
package config_test
import (
"encoding/json"
"net/url"
"testing"
"github.com/9seconds/mtg/v2/internal/config"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/suite"
)
type typeHttpsURLTestStruct struct {
Value config.TypeHttpsURL `json:"value"`
}
type HttpsURLTestSuite struct {
suite.Suite
}
func (suite *HttpsURLTestSuite) TestUnmarshalFail() {
testData := []string{
"",
"https://",
"://lala",
"/path",
"http://example.com",
"socks5://example.com",
}
for _, v := range testData {
data, err := json.Marshal(map[string]string{
"value": v,
})
suite.NoError(err)
suite.T().Run(v, func(t *testing.T) {
assert.Error(t, json.Unmarshal(data, &typeHttpsURLTestStruct{}))
})
}
}
func (suite *HttpsURLTestSuite) TestUnmarshalOk() {
testData := map[string]string{
"https://example.com": "https://example.com",
"https://example.com:8443": "https://example.com:8443",
"https://example.com/path?q=1": "https://example.com/path?q=1",
"https://user:pass@example.com": "https://user:pass@example.com",
}
for k, v := range testData {
value := v
data, err := json.Marshal(map[string]string{
"value": k,
})
suite.NoError(err)
suite.T().Run(k, func(t *testing.T) {
testStruct := &typeHttpsURLTestStruct{}
assert.NoError(t, json.Unmarshal(data, testStruct))
parsed, _ := url.Parse(value)
assert.Equal(t, parsed.Scheme, testStruct.Value.Get(nil).Scheme)
assert.Equal(t, parsed.Host, testStruct.Value.Get(nil).Host)
assert.Equal(t, parsed.RawQuery, testStruct.Value.Get(nil).RawQuery)
assert.Equal(t, parsed.Path, testStruct.Value.Get(nil).Path)
})
}
}
func (suite *HttpsURLTestSuite) TestMarshalOk() {
parsed, _ := url.Parse("https://example.com/path?q=1")
testStruct := &typeHttpsURLTestStruct{
Value: config.TypeHttpsURL{
Value: parsed,
},
}
encodedJSON, err := json.Marshal(testStruct)
suite.NoError(err)
suite.JSONEq(`{"value": "https://example.com/path?q=1"}`,
string(encodedJSON))
}
func (suite *HttpsURLTestSuite) TestGet() {
emptyURL := &url.URL{}
value := config.TypeHttpsURL{}
suite.Equal(emptyURL, value.Get(emptyURL))
value.Value = &url.URL{}
suite.Equal(value.Value, value.Get(emptyURL))
}
func TestTypeHttpsURL(t *testing.T) {
t.Parallel()
suite.Run(t, &HttpsURLTestSuite{})
}
+5
View File
@@ -2,6 +2,7 @@ package testlib
import (
"context"
"net"
"net/http"
"github.com/9seconds/mtg/v2/essentials"
@@ -24,6 +25,10 @@ func (m *MtglibNetworkMock) DialContext(ctx context.Context, network, address st
return args.Get(0).(essentials.Conn), args.Error(1) //nolint: wrapcheck, forcetypeassert
}
func (m *MtglibNetworkMock) NativeDialer() *net.Dialer {
return m.Called().Get(0).(*net.Dialer)
}
func (m *MtglibNetworkMock) MakeHTTPClient(dialFunc func(ctx context.Context,
network, address string) (essentials.Conn, error),
) *http.Client {
+17
View File
@@ -14,6 +14,23 @@ import (
)
func main() {
// this runs profiling server. To enable it, build with prof tag
// $ go build -tags prof
//
// Then you can pass a port using MTG_PROF_PORT environment variable.
// Default is 6000
// $ MTG_PROF_PORT=6000 mtg run config.toml
//
// It will run a webserver with profiling data on
// localhost:${MTG_PROF_PORT:-6000}.
//
// To collect PGO do following:
// $ curl -o default.pgo 'http://localhost:6000/debug/pprof/profile?seconds=300'
//
// See also https://pkg.go.dev/net/http/pprof
// https://go.dev/blog/pprof
runProfile()
cli := &cli.CLI{}
ctx := kong.Parse(cli, kong.Vars{
"version": getVersion(),
+97 -18
View File
@@ -1,11 +1,36 @@
# @generated - this file is auto-generated by `mise lock` https://mise.jdx.dev/dev-tools/mise-lock.html
[[tools.go]]
version = "1.26.0"
version = "1.26.1"
backend = "core:go"
"platforms.linux-arm64" = { checksum = "sha256:bd03b743eb6eb4193ea3c3fd3956546bf0e3ca5b7076c8226334afe6b75704cd", url = "https://dl.google.com/go/go1.26.0.linux-arm64.tar.gz"}
"platforms.linux-x64" = { checksum = "sha256:aac1b08a0fb0c4e0a7c1555beb7b59180b05dfc5a3d62e40e9de90cd42f88235", url = "https://dl.google.com/go/go1.26.0.linux-amd64.tar.gz"}
"platforms.macos-arm64" = { checksum = "sha256:b1640525dfe68f066d56f200bef7bf4dce955a1a893bd061de6754c211431023", url = "https://dl.google.com/go/go1.26.0.darwin-arm64.tar.gz"}
"platforms.macos-x64" = { checksum = "sha256:1ca28b7703cbea05a65b2a1d92d6b308610ef92f8824578a0874f2e60c9d5a22", url = "https://dl.google.com/go/go1.26.0.darwin-amd64.tar.gz"}
"platforms.windows-x64" = { checksum = "sha256:9bbe0fc64236b2b51f6255c05c4232532b8ecc0e6d2e00950bd3021d8a4d07d4", url = "https://dl.google.com/go/go1.26.0.windows-amd64.zip"}
[tools.go."platforms.linux-arm64"]
checksum = "sha256:a290581cfe4fe28ddd737dde3095f3dbeb7f2e4065cab4eae44dfc53b760c2f7"
url = "https://dl.google.com/go/go1.26.1.linux-arm64.tar.gz"
[tools.go."platforms.linux-arm64-musl"]
checksum = "sha256:a290581cfe4fe28ddd737dde3095f3dbeb7f2e4065cab4eae44dfc53b760c2f7"
url = "https://dl.google.com/go/go1.26.1.linux-arm64.tar.gz"
[tools.go."platforms.linux-x64"]
checksum = "sha256:031f088e5d955bab8657ede27ad4e3bc5b7c1ba281f05f245bcc304f327c987a"
url = "https://dl.google.com/go/go1.26.1.linux-amd64.tar.gz"
[tools.go."platforms.linux-x64-musl"]
checksum = "sha256:031f088e5d955bab8657ede27ad4e3bc5b7c1ba281f05f245bcc304f327c987a"
url = "https://dl.google.com/go/go1.26.1.linux-amd64.tar.gz"
[tools.go."platforms.macos-arm64"]
checksum = "sha256:353df43a7811ce284c8938b5f3c7df40b7bfb6f56cb165b150bc40b5e2dd541f"
url = "https://dl.google.com/go/go1.26.1.darwin-arm64.tar.gz"
[tools.go."platforms.macos-x64"]
checksum = "sha256:65773dab2f8cc4cd23d93ba6d0a805de150ca0b78378879292be0b903b8cdd08"
url = "https://dl.google.com/go/go1.26.1.darwin-amd64.tar.gz"
[tools.go."platforms.windows-x64"]
checksum = "sha256:9b68112c913f45b7aebbf13c036721264bbba7e03a642f8f7490c561eebd1ecc"
url = "https://dl.google.com/go/go1.26.1.windows-amd64.zip"
[[tools."go:golang.org/x/pkgsite/cmd/pkgsite"]]
version = "latest"
@@ -24,19 +49,73 @@ version = "0.9.2"
backend = "go:mvdan.cc/gofumpt"
[[tools.golangci-lint]]
version = "2.10.1"
version = "2.11.4"
backend = "aqua:golangci/golangci-lint"
"platforms.linux-arm64" = { checksum = "sha256:6652b42ae02915eb2f9cb2a2e0cac99514c8eded8388d88ae3e06e1a52c00de8", url = "https://github.com/golangci/golangci-lint/releases/download/v2.10.1/golangci-lint-2.10.1-linux-arm64.tar.gz"}
"platforms.linux-x64" = { checksum = "sha256:dfa775874cf0561b404a02a8f4481fc69b28091da95aa697259820d429b09c99", url = "https://github.com/golangci/golangci-lint/releases/download/v2.10.1/golangci-lint-2.10.1-linux-amd64.tar.gz"}
"platforms.macos-arm64" = { checksum = "sha256:03bfadf67e52b441b7ec21305e501c717df93c959836d66c7f97312654acb297", url = "https://github.com/golangci/golangci-lint/releases/download/v2.10.1/golangci-lint-2.10.1-darwin-arm64.tar.gz"}
"platforms.macos-x64" = { checksum = "sha256:66fb0da81b8033b477f97eea420d4b46b230ca172b8bb87c6610109f3772b6b6", url = "https://github.com/golangci/golangci-lint/releases/download/v2.10.1/golangci-lint-2.10.1-darwin-amd64.tar.gz"}
"platforms.windows-x64" = { checksum = "sha256:c60c87695e79db8e320f0e5be885059859de52bb5ee5f11be5577828570bc2a3", url = "https://github.com/golangci/golangci-lint/releases/download/v2.10.1/golangci-lint-2.10.1-windows-amd64.zip"}
[tools.golangci-lint."platforms.linux-arm64"]
checksum = "sha256:3bcfa2e6f3d32b2bf5cd75eaa876447507025e0303698633f722a05331988db4"
url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-linux-arm64.tar.gz"
[tools.golangci-lint."platforms.linux-arm64-musl"]
checksum = "sha256:3bcfa2e6f3d32b2bf5cd75eaa876447507025e0303698633f722a05331988db4"
url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-linux-arm64.tar.gz"
[tools.golangci-lint."platforms.linux-x64"]
checksum = "sha256:200c5b7503f67b59a6743ccf32133026c174e272b930ee79aa2aa6f37aca7ef1"
url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-linux-amd64.tar.gz"
[tools.golangci-lint."platforms.linux-x64-musl"]
checksum = "sha256:200c5b7503f67b59a6743ccf32133026c174e272b930ee79aa2aa6f37aca7ef1"
url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-linux-amd64.tar.gz"
[tools.golangci-lint."platforms.macos-arm64"]
checksum = "sha256:02db2a2dae8b26812e53b0688a6f617e3ef1f489790e829ea22862cf76945675"
url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-darwin-arm64.tar.gz"
provenance = "github-attestations"
[tools.golangci-lint."platforms.macos-x64"]
checksum = "sha256:c900d4048db75d1edfd550fd11cf6a9b3008e7caa8e119fcddbc700412d63e60"
url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-darwin-amd64.tar.gz"
[tools.golangci-lint."platforms.windows-x64"]
checksum = "sha256:4932cfca5e75bf60fe1c576edf459e5e809e6644664a068185d64b84af3fad9e"
url = "https://github.com/golangci/golangci-lint/releases/download/v2.11.4/golangci-lint-2.11.4-windows-amd64.zip"
[[tools.goreleaser]]
version = "2.14.1"
version = "2.15.2"
backend = "aqua:goreleaser/goreleaser"
"platforms.linux-arm64" = { checksum = "sha256:a84d3b27f052c12ad5c8342d7caf1450a7174a305730aed21d72db09301e49a5", url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.1/goreleaser_Linux_arm64.tar.gz"}
"platforms.linux-x64" = { checksum = "sha256:2df975a7acbfdeaf888d596cab0024d48ec7fb7d747e1d08b90948b791f40a5f", url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.1/goreleaser_Linux_x86_64.tar.gz"}
"platforms.macos-arm64" = { checksum = "sha256:9f2e47f847b4f4177376fc6aa6914fbc7f673f59720076747e738b578c2e896e", url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.1/goreleaser_Darwin_all.tar.gz"}
"platforms.macos-x64" = { checksum = "sha256:9f2e47f847b4f4177376fc6aa6914fbc7f673f59720076747e738b578c2e896e", url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.1/goreleaser_Darwin_all.tar.gz"}
"platforms.windows-x64" = { checksum = "sha256:d7a3d8ba795e97ab8c4f8003630d300da164adf21fde5a4049440c20f15c3137", url = "https://github.com/goreleaser/goreleaser/releases/download/v2.14.1/goreleaser_Windows_x86_64.zip"}
[tools.goreleaser."platforms.linux-arm64"]
checksum = "sha256:5db66761a98f6693161e49e1a95d28d2673a892ba60cb4a5e16736cafd41c4c9"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_arm64.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.linux-arm64-musl"]
checksum = "sha256:5db66761a98f6693161e49e1a95d28d2673a892ba60cb4a5e16736cafd41c4c9"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_arm64.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.linux-x64"]
checksum = "sha256:0ebdbf0353aba566b969dde746cc4e4806f96c27aa2f3971b229a9df7611fedc"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_x86_64.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.linux-x64-musl"]
checksum = "sha256:0ebdbf0353aba566b969dde746cc4e4806f96c27aa2f3971b229a9df7611fedc"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Linux_x86_64.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.macos-arm64"]
checksum = "sha256:0e6bd67688ac949780bf1166813a91f89856898ef4c40d7d46c2c74ebaa4b9ee"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Darwin_all.tar.gz"
provenance = "github-attestations"
[tools.goreleaser."platforms.macos-x64"]
checksum = "sha256:0e6bd67688ac949780bf1166813a91f89856898ef4c40d7d46c2c74ebaa4b9ee"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Darwin_all.tar.gz"
provenance = "cosign"
[tools.goreleaser."platforms.windows-x64"]
checksum = "sha256:7459832946dbe122c144f8d7f87484d8572ca005b779310aa6bb03346e8de17a"
url = "https://github.com/goreleaser/goreleaser/releases/download/v2.15.2/goreleaser_Windows_x86_64.zip"
provenance = "cosign"
+64
View File
@@ -3,9 +3,12 @@ package mtglib
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"net"
"sync/atomic"
"time"
"github.com/9seconds/mtg/v2/essentials"
"github.com/pires/go-proxyproto"
@@ -95,3 +98,64 @@ func newConnProxyProtocol(source, target essentials.Conn) *connProxyProtocol {
sourceAddr: source.RemoteAddr(),
}
}
// idleTracker is a shared idle tracker for a pair of relay connections.
// Both directions update the same timestamp so that activity in one direction
// prevents the other (idle) direction from timing out.
type idleTracker struct {
lastActive atomic.Pointer[time.Time]
timeout time.Duration
}
func newIdleTracker(timeout time.Duration) *idleTracker {
t := &idleTracker{timeout: timeout}
t.touch()
return t
}
func (t *idleTracker) touch() {
stamp := time.Now()
t.lastActive.Store(&stamp)
}
func (t *idleTracker) isIdle() bool {
return time.Since(*t.lastActive.Load()) >= t.timeout
}
type connIdleTimeout struct {
essentials.Conn
tracker *idleTracker
}
func (c connIdleTimeout) Read(b []byte) (int, error) {
var netErr net.Error
for {
c.SetReadDeadline(time.Now().Add(c.tracker.timeout)) //nolint: errcheck
n, err := c.Conn.Read(b)
switch {
case err == nil:
c.tracker.touch()
return n, nil
case errors.As(err, &netErr) && netErr.Timeout() && !c.tracker.isIdle():
continue
}
return n, err
}
}
func (c connIdleTimeout) Write(b []byte) (int, error) {
c.SetWriteDeadline(time.Now().Add(c.tracker.timeout)) //nolint: errcheck
n, err := c.Conn.Write(b)
if n > 0 {
c.tracker.touch()
}
return n, err //nolint: wrapcheck
}
+151
View File
@@ -16,6 +16,12 @@ import (
"github.com/stretchr/testify/suite"
)
type netTimeoutError struct{}
func (e netTimeoutError) Error() string { return "i/o timeout" }
func (e netTimeoutError) Timeout() bool { return true }
func (e netTimeoutError) Temporary() bool { return true }
type ConnRewindBaseConn struct {
testlib.EssentialsConnMock
@@ -291,6 +297,141 @@ func (suite *ConnProxyProtocolTestSuite) TearDownTest() {
suite.targetConnMock.AssertExpectations(suite.T())
}
type IdleTrackerTestSuite struct {
suite.Suite
}
func (suite *IdleTrackerTestSuite) TestNewNotIdle() {
tracker := newIdleTracker(time.Second)
suite.False(tracker.isIdle())
}
func (suite *IdleTrackerTestSuite) TestIdleAfterTimeout() {
tracker := newIdleTracker(10 * time.Millisecond)
time.Sleep(20 * time.Millisecond)
suite.True(tracker.isIdle())
}
func (suite *IdleTrackerTestSuite) TestTouchResetsIdle() {
tracker := newIdleTracker(50 * time.Millisecond)
time.Sleep(30 * time.Millisecond)
tracker.touch()
suite.False(tracker.isIdle())
}
type ConnIdleTimeoutTestSuite struct {
suite.Suite
connMock *testlib.EssentialsConnMock
tracker *idleTracker
conn connIdleTimeout
}
func (suite *ConnIdleTimeoutTestSuite) SetupTest() {
suite.connMock = &testlib.EssentialsConnMock{}
suite.tracker = newIdleTracker(time.Second)
suite.conn = connIdleTimeout{
Conn: suite.connMock,
tracker: suite.tracker,
}
}
func (suite *ConnIdleTimeoutTestSuite) TearDownTest() {
suite.connMock.AssertExpectations(suite.T())
}
func (suite *ConnIdleTimeoutTestSuite) TestReadOk() {
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(5, nil)
n, err := suite.conn.Read(make([]byte, 10))
suite.NoError(err)
suite.Equal(5, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestReadNonTimeoutErr() {
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, io.EOF)
n, err := suite.conn.Read(make([]byte, 10))
suite.True(errors.Is(err, io.EOF))
suite.Equal(0, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestReadTimeoutRetriesWhenNotIdle() {
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, netTimeoutError{})
suite.connMock.On("Read", mock.Anything).Once().Return(5, nil)
n, err := suite.conn.Read(make([]byte, 10))
suite.NoError(err)
suite.Equal(5, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestReadTimeoutClosesWhenIdle() {
suite.tracker = newIdleTracker(time.Millisecond)
suite.conn = connIdleTimeout{
Conn: suite.connMock,
tracker: suite.tracker,
}
time.Sleep(5 * time.Millisecond)
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, netTimeoutError{})
n, err := suite.conn.Read(make([]byte, 10))
suite.Equal(0, n)
netErr, ok := err.(net.Error) //nolint: errorlint
suite.True(ok)
suite.True(netErr.Timeout())
}
func (suite *ConnIdleTimeoutTestSuite) TestSharedTrackerPreventsFalseTimeout() {
connMock2 := &testlib.EssentialsConnMock{}
conn2 := connIdleTimeout{
Conn: connMock2,
tracker: suite.tracker,
}
connMock2.On("SetWriteDeadline", mock.Anything).Return(nil)
connMock2.On("Write", mock.Anything).Once().Return(5, nil)
_, _ = conn2.Write(make([]byte, 5))
suite.connMock.On("SetReadDeadline", mock.Anything).Return(nil)
suite.connMock.On("Read", mock.Anything).Once().Return(0, netTimeoutError{})
suite.connMock.On("Read", mock.Anything).Once().Return(3, nil)
n, err := suite.conn.Read(make([]byte, 10))
suite.NoError(err)
suite.Equal(3, n)
connMock2.AssertExpectations(suite.T())
}
func (suite *ConnIdleTimeoutTestSuite) TestWriteOk() {
suite.connMock.On("SetWriteDeadline", mock.Anything).Return(nil)
suite.connMock.On("Write", mock.Anything).Once().Return(5, nil)
n, err := suite.conn.Write(make([]byte, 5))
suite.NoError(err)
suite.Equal(5, n)
}
func (suite *ConnIdleTimeoutTestSuite) TestWriteErr() {
suite.connMock.On("SetWriteDeadline", mock.Anything).Return(nil)
suite.connMock.On("Write", mock.Anything).Once().Return(0, io.EOF)
n, err := suite.conn.Write(make([]byte, 5))
suite.True(errors.Is(err, io.EOF))
suite.Equal(0, n)
}
func TestConnTraffic(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnTrafficTestSuite{})
@@ -305,3 +446,13 @@ func TestConnProxyProtocol(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnProxyProtocolTestSuite{})
}
func TestIdleTracker(t *testing.T) {
t.Parallel()
suite.Run(t, &IdleTrackerTestSuite{})
}
func TestConnIdleTimeout(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnIdleTimeoutTestSuite{})
}
+19 -3
View File
@@ -77,8 +77,13 @@ const (
// DefaultIdleTimeout is a default timeout for closing a connection in case of
// idling.
//
// Deprecated: no longer in use because of changed TCP relay algorithm.
DefaultIdleTimeout = time.Minute
// Set to 5 minutes to survive typical mobile sleep periods (2-5 min) and
// avoid racing with MTProto ping_delay_disconnect (~60s interval).
DefaultIdleTimeout = 5 * time.Minute
// DefaultHandshakeTimeout defines a time period during which the
// all handshake ceremonies must be completed.
DefaultHandshakeTimeout = 10 * time.Second
// DefaultTolerateTimeSkewness is a default timeout for time skewness on a
// faketls timeout verification.
@@ -99,6 +104,13 @@ const (
// reads from Telegram after which connection will be terminated. This is
// required to abort stale connections.
TCPRelayReadTimeout = 20 * time.Second
// DoppelGangerPerRaid defines a number of requests to each URL
// per raid.
DoppelGangerPerRaid = 10
// DoppelGangerEach defines a time period between each crawl attempt.
DoppelGangerEach = 6 * time.Hour
)
// Network defines a knowledge how to work with a network. It may sound fun but
@@ -117,13 +129,17 @@ type Network interface {
// Dial establishes context-free TCP connections.
Dial(network, address string) (essentials.Conn, error)
// DialContext dials using a context. This is a preferrable way of
// DialContext dials using a context. This is a preferable way of
// establishing TCP connections.
DialContext(ctx context.Context, network, address string) (essentials.Conn, error)
// MakeHTTPClient build an HTTP client with given dial function. If nothing is
// provided, then DialContext of this interface is going to be used.
MakeHTTPClient(func(ctx context.Context, network, address string) (essentials.Conn, error)) *http.Client
// NativeDialer returns a configured instance of native dialer that
// skips proxy connections or any other irrelevant settings.
NativeDialer() *net.Dialer
}
// AntiReplayCache is an interface that is used to detect replay attacks based
+36 -43
View File
@@ -2,7 +2,10 @@ package dc
import (
"context"
"net"
"time"
"github.com/9seconds/mtg/v2/essentials"
)
type preferIP uint8
@@ -39,46 +42,36 @@ type Updater interface {
}
// https://github.com/telegramdesktop/tdesktop/blob/master/Telegram/SourceFiles/mtproto/mtproto_dc_options.cpp#L30
var defaultDCAddrSet = dcAddrSet{
v4: map[int][]Addr{
1: {
{Network: "tcp4", Address: "149.154.175.50:443"},
},
2: {
{Network: "tcp4", Address: "149.154.167.51:443"},
{Network: "tcp4", Address: "95.161.76.100:443"},
},
3: {
{Network: "tcp4", Address: "149.154.175.100:443"},
},
4: {
{Network: "tcp4", Address: "149.154.167.91:443"},
},
5: {
{Network: "tcp4", Address: "149.154.171.5:443"},
},
203: {
{Network: "tcp4", Address: "91.105.192.100:443"},
},
},
v6: map[int][]Addr{
1: {
{Network: "tcp6", Address: "[2001:b28:f23d:f001::a]:443"},
},
2: {
{Network: "tcp6", Address: "[2001:67c:04e8:f002::a]:443"},
},
3: {
{Network: "tcp6", Address: "[2001:b28:f23d:f003::a]:443"},
},
4: {
{Network: "tcp6", Address: "[2001:67c:04e8:f004::a]:443"},
},
5: {
{Network: "tcp6", Address: "[2001:b28:f23f:f005::a]:443"},
},
203: {
{Network: "tcp6", Address: "[2a0a:f280:0203:000a:5000:0000:0000:0100]:443"},
},
},
}
var defaultDCAddrSet = (func() dcAddrSet {
addrSet := dcAddrSet{
v4: make(map[int][]Addr),
v6: make(map[int][]Addr),
}
for dcid, ips := range essentials.TelegramCoreAddresses {
for _, addr := range ips {
host, _, err := net.SplitHostPort(addr)
if err != nil {
panic(err)
}
ip := net.ParseIP(host)
if ip == nil {
panic(addr)
}
if ip.To4() == nil {
addrSet.v6[dcid] = append(addrSet.v6[dcid], Addr{
Network: "tcp6",
Address: addr,
})
} else {
addrSet.v4[dcid] = append(addrSet.v4[dcid], Addr{
Network: "tcp4",
Address: addr,
})
}
}
}
return addrSet
})()
+1 -1
View File
@@ -20,7 +20,7 @@ func (t *Telegram) GetAddresses(dc int) []Addr {
case preferIPOnlyIPv4:
return t.view.getV4(dc)
case preferIPOnlyIPv6:
return t.view.getV4(dc)
return t.view.getV6(dc)
case preferIPPreferIPv4:
return append(t.view.getV4(dc), t.view.getV6(dc)...)
}
+6 -2
View File
@@ -5,15 +5,19 @@ type dcView struct {
}
func (d dcView) getV4(dc int) []Addr {
addrs := d.publicConfigs.getV4(dc)
var addrs []Addr
addrs = append(addrs, defaultDCAddrSet.getV4(dc)...)
addrs = append(addrs, d.publicConfigs.getV4(dc)...)
return addrs
}
func (d dcView) getV6(dc int) []Addr {
addrs := d.publicConfigs.getV6(dc)
var addrs []Addr
addrs = append(addrs, defaultDCAddrSet.getV6(dc)...)
addrs = append(addrs, d.publicConfigs.getV6(dc)...)
return addrs
}
+118
View File
@@ -0,0 +1,118 @@
package doppel
import (
"bytes"
"context"
"sync"
"time"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
var doppelBufPool = sync.Pool{
New: func() any {
b := make([]byte, tls.MaxRecordSize)
return &b
},
}
type Conn struct {
essentials.Conn
p *connPayload
}
type connPayload struct {
ctx context.Context
ctxCancel context.CancelCauseFunc
stats Stats
wg sync.WaitGroup
writeStream bytes.Buffer
writtenCond sync.Cond
done bool
}
func (c Conn) Write(p []byte) (int, error) {
if len(p) == 0 {
return 0, context.Cause(c.p.ctx)
}
c.p.writtenCond.L.Lock()
c.p.writeStream.Write(p)
c.p.writtenCond.L.Unlock()
c.p.writtenCond.Signal()
return len(p), context.Cause(c.p.ctx)
}
func (c Conn) start() {
bp := doppelBufPool.Get().(*[]byte)
buf := *bp
defer doppelBufPool.Put(bp)
timer := time.NewTimer(c.p.stats.Delay())
defer timer.Stop()
for {
select {
case <-c.p.ctx.Done():
return
case <-timer.C:
timer.Reset(c.p.stats.Delay())
}
size := c.p.stats.Size()
c.p.writtenCond.L.Lock()
for c.p.writeStream.Len() == 0 && !c.p.done {
c.p.writtenCond.Wait()
}
n, _ := c.p.writeStream.Read(buf[tls.SizeHeader : tls.SizeHeader+size])
c.p.writtenCond.L.Unlock()
if n == 0 {
continue
}
if err := tls.WriteRecordInPlace(c.Conn, buf, n); err != nil {
c.p.ctxCancel(err)
return
}
}
}
func (c Conn) Stop() {
c.p.ctxCancel(nil)
c.p.writtenCond.L.Lock()
c.p.done = true
c.p.writtenCond.L.Unlock()
c.p.writtenCond.Broadcast()
c.p.wg.Wait()
}
func NewConn(ctx context.Context, conn essentials.Conn, stats Stats) Conn {
ctx, cancel := context.WithCancelCause(ctx)
rv := Conn{
Conn: conn,
p: &connPayload{
ctx: ctx,
ctxCancel: cancel,
stats: stats,
writtenCond: sync.Cond{
L: &sync.Mutex{},
},
},
}
rv.p.writeStream.Grow(tls.DefaultBufferSize)
rv.p.wg.Go(func() {
rv.start()
})
return rv
}
+194
View File
@@ -0,0 +1,194 @@
package doppel
import (
"bytes"
"context"
"encoding/binary"
"errors"
"io"
"sync"
"testing"
"time"
"github.com/9seconds/mtg/v2/internal/testlib"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/suite"
)
type ConnMock struct {
testlib.EssentialsConnMock
mu sync.Mutex
writeBuffer bytes.Buffer
}
func (m *ConnMock) Write(p []byte) (int, error) {
args := m.Called(p)
if err := args.Error(1); err != nil {
return args.Int(0), err
}
m.mu.Lock()
defer m.mu.Unlock()
return m.writeBuffer.Write(p)
}
func (m *ConnMock) Written() []byte {
m.mu.Lock()
defer m.mu.Unlock()
return bytes.Clone(m.writeBuffer.Bytes())
}
type ConnTestSuite struct {
suite.Suite
connMock *ConnMock
ctx context.Context
ctxCancel context.CancelFunc
}
func (suite *ConnTestSuite) SetupTest() {
ctx, cancel := context.WithCancel(context.Background())
suite.ctx = ctx
suite.ctxCancel = cancel
suite.connMock = &ConnMock{}
}
func (suite *ConnTestSuite) TearDownTest() {
suite.ctxCancel()
suite.connMock.AssertExpectations(suite.T())
}
func (suite *ConnTestSuite) makeConn() Conn {
return NewConn(suite.ctx, suite.connMock, Stats{
k: 2.0,
lambda: 0.01,
})
}
func (suite *ConnTestSuite) TestWriteBuffersData() {
suite.connMock.
On("Write", mock.AnythingOfType("[]uint8")).
Return(0, nil).
Maybe()
c := suite.makeConn()
defer c.Stop()
n, err := c.Write([]byte{1, 2, 3})
suite.NoError(err)
suite.Equal(3, n)
}
func (suite *ConnTestSuite) TestWriteOutputsTLSRecords() {
suite.connMock.
On("Write", mock.AnythingOfType("[]uint8")).
Return(0, nil).
Maybe()
c := suite.makeConn()
payload := []byte("hello doppelganger")
_, err := c.Write(payload)
suite.NoError(err)
suite.Eventually(func() bool {
return len(suite.connMock.Written()) > 0
}, 2*time.Second, time.Millisecond)
c.Stop()
assembled := &bytes.Buffer{}
reader := bytes.NewReader(suite.connMock.Written())
for {
header := make([]byte, tls.SizeHeader)
if _, err := io.ReadFull(reader, header); err != nil {
break
}
suite.Equal(byte(tls.TypeApplicationData), header[0])
suite.Equal(tls.TLSVersion[:], header[tls.SizeRecordType:tls.SizeRecordType+tls.SizeVersion])
length := binary.BigEndian.Uint16(header[tls.SizeRecordType+tls.SizeVersion:])
suite.Greater(length, uint16(0))
rec := make([]byte, length)
_, err := io.ReadFull(reader, rec)
suite.NoError(err)
assembled.Write(rec)
}
suite.Equal(payload, assembled.Bytes())
}
func (suite *ConnTestSuite) TestWriteReturnsErrorAfterStop() {
suite.connMock.
On("Write", mock.AnythingOfType("[]uint8")).
Return(0, nil).
Maybe()
c := suite.makeConn()
c.Stop()
time.Sleep(10 * time.Millisecond)
_, err := c.Write([]byte{1})
suite.Error(err)
}
func (suite *ConnTestSuite) TestStopDoesNotDeadlockWhenStartIsWaiting() {
suite.connMock.
On("Write", mock.AnythingOfType("[]uint8")).
Return(0, nil).
Maybe()
for range 100 {
func() {
ctx, cancel := context.WithCancel(suite.ctx)
defer cancel()
c := NewConn(ctx, suite.connMock, Stats{
k: 2.0,
lambda: 0.01,
})
done := make(chan struct{})
go func() {
defer close(done)
c.Stop()
}()
select {
case <-done:
case <-time.After(2 * time.Second):
suite.Fail("Stop() deadlocked: start() likely stuck in writtenCond.Wait()")
}
}()
}
}
func (suite *ConnTestSuite) TestStopOnUnderlyingWriteError() {
suite.connMock.
On("Write", mock.AnythingOfType("[]uint8")).
Return(0, errors.New("connection reset")).
Maybe()
c := suite.makeConn()
_, _ = c.Write([]byte("data"))
suite.Eventually(func() bool {
_, err := c.Write([]byte{1})
return err != nil
}, 2*time.Second, time.Millisecond)
}
func TestConn(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnTestSuite{})
}
+267
View File
@@ -0,0 +1,267 @@
package doppel
import (
"context"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/9seconds/mtg/v2/essentials"
)
const (
DoppelGangerMaxDurations = 4096
DoppelGangerScoutRaidEach = 6 * time.Hour
DoppelGangerScoutRepeats = 10
MinCertSizesToCalculate = 3
)
// NoiseParams holds the measured cert chain size for FakeTLS noise calibration.
// If Mean is 0, the caller should use a legacy fallback.
type NoiseParams struct {
Mean int
Jitter int
}
type scoutRaidResult struct {
durations []time.Duration
certSizes []int
}
type gangerConnRequest struct {
ret chan<- Conn
payload essentials.Conn
}
type Ganger struct {
ctx context.Context
ctxCancel context.CancelFunc
logger Logger
wg sync.WaitGroup
scout Scout
scoutRaidEach time.Duration
scoutRaidRepeats int
drs bool
stats Stats
durations []time.Duration
certSizes []int
noiseParams atomic.Pointer[NoiseParams]
connRequests chan gangerConnRequest
}
func (g *Ganger) Shutdown() {
g.ctxCancel()
g.wg.Wait()
}
func (g *Ganger) Run() {
g.wg.Go(func() {
g.run()
})
}
// NoiseParams returns the current cert-size-based noise parameters.
// Returns zero-value NoiseParams if not yet measured (caller should use fallback).
func (g *Ganger) NoiseParams() NoiseParams {
if p := g.noiseParams.Load(); p != nil {
return *p
}
return NoiseParams{}
}
func (g *Ganger) NewConn(conn essentials.Conn) (Conn, error) {
rvChan := make(chan Conn)
req := gangerConnRequest{
ret: rvChan,
payload: conn,
}
defer close(req.ret)
select {
case <-g.ctx.Done():
return Conn{}, context.Cause(g.ctx)
case g.connRequests <- req:
}
select {
case <-g.ctx.Done():
return Conn{}, context.Cause(g.ctx)
case conn := <-rvChan:
return conn, nil
}
}
func (g *Ganger) run() {
scoutTicker := time.NewTicker(g.scoutRaidEach)
defer func() {
scoutTicker.Stop()
select {
case <-scoutTicker.C:
default:
}
}()
scoutCollectedChan := make(chan scoutRaidResult)
currentScoutCollectedChan := scoutCollectedChan
updatedStatsChan := make(chan Stats)
g.wg.Go(func() {
g.runScoutRaid(scoutCollectedChan)
})
for {
select {
case <-g.ctx.Done():
return
case result := <-currentScoutCollectedChan:
g.durations = append(g.durations, result.durations...)
if len(g.durations) > DoppelGangerMaxDurations {
copy(g.durations, g.durations[len(g.durations)-DoppelGangerMaxDurations:])
g.durations = g.durations[:DoppelGangerMaxDurations]
}
// Update cert sizes and recompute noise params.
g.certSizes = append(g.certSizes, result.certSizes...)
if len(g.certSizes) > DoppelGangerMaxDurations {
g.certSizes = g.certSizes[len(g.certSizes)-DoppelGangerMaxDurations:]
}
if len(g.certSizes) >= MinCertSizesToCalculate {
g.updateNoiseParams()
}
if len(g.durations) < MinDurationsToCalculate {
continue
}
durations := g.durations
currentScoutCollectedChan = nil
g.wg.Go(func() {
select {
case <-g.ctx.Done():
case updatedStatsChan <- NewStats(durations, g.drs):
}
})
case stats := <-updatedStatsChan:
g.stats = stats
currentScoutCollectedChan = scoutCollectedChan
case <-scoutTicker.C:
g.wg.Go(func() {
g.runScoutRaid(scoutCollectedChan)
})
case req := <-g.connRequests:
select {
case <-g.ctx.Done():
case req.ret <- NewConn(g.ctx, req.payload, g.stats):
}
}
}
}
func (g *Ganger) updateNoiseParams() {
if len(g.certSizes) == 0 {
return
}
sum := 0
for _, s := range g.certSizes {
sum += s
}
mean := sum / len(g.certSizes)
maxDev := 0
for _, s := range g.certSizes {
d := s - mean
if d < 0 {
d = -d
}
if d > maxDev {
maxDev = d
}
}
if maxDev < 100 {
maxDev = 100
}
np := &NoiseParams{Mean: mean, Jitter: maxDev}
g.noiseParams.Store(np)
g.logger.Info(fmt.Sprintf(
"updated noise params: mean=%d jitter=%d samples=%d",
mean, maxDev, len(g.certSizes),
))
}
func (g *Ganger) runScoutRaid(rvChan chan<- scoutRaidResult) {
var result scoutRaidResult
for range g.scoutRaidRepeats {
learned, err := g.scout.Learn(g.ctx)
if err != nil {
g.logger.WarningError("cannot learn", err)
continue
}
result.durations = append(result.durations, learned.Durations...)
if learned.CertSize > 0 {
result.certSizes = append(result.certSizes, learned.CertSize)
}
}
select {
case <-g.ctx.Done():
return
case rvChan <- result:
}
}
func NewGanger(
ctx context.Context,
network Network,
logger Logger,
scoutEach time.Duration,
scoutRepeats int,
urls []string,
drs bool,
) *Ganger {
ctx, cancel := context.WithCancel(ctx)
if scoutEach == 0 {
scoutEach = DoppelGangerScoutRaidEach
}
if scoutRepeats == 0 {
scoutRepeats = DoppelGangerScoutRepeats
}
return &Ganger{
ctx: ctx,
ctxCancel: cancel,
logger: logger,
scoutRaidEach: scoutEach,
scoutRaidRepeats: scoutRepeats,
drs: drs,
stats: Stats{
k: StatsDefaultK,
lambda: StatsDefaultLambda,
drs: drs,
},
scout: NewScout(network, urls),
connRequests: make(chan gangerConnRequest),
}
}
+107
View File
@@ -0,0 +1,107 @@
package doppel
import (
"bytes"
"sync"
"testing"
"time"
"github.com/9seconds/mtg/v2/internal/testlib"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/suite"
)
type GangerTestSuite struct {
TLSServerTestSuite
log *LoggerMock
g *Ganger
}
func (suite *GangerTestSuite) SetupTest() {
suite.TLSServerTestSuite.SetupTest()
suite.log = &LoggerMock{}
suite.log.
On("Info", mock.AnythingOfType("string")).
Maybe()
suite.log.
On("WarningError", mock.AnythingOfType("string"), mock.Anything).
Maybe()
suite.g = NewGanger(suite.ctx, suite.network, suite.log, time.Hour, 1, suite.urls, true)
suite.g.Run()
}
func (suite *GangerTestSuite) TearDownTest() {
suite.g.Shutdown()
suite.log.AssertExpectations(suite.T())
suite.TLSServerTestSuite.TearDownTest()
}
func (suite *GangerTestSuite) TestNewConnAfterShutdown() {
suite.g.Shutdown()
connMock := &testlib.EssentialsConnMock{}
_, err := suite.g.NewConn(connMock)
suite.Error(err)
}
func (suite *GangerTestSuite) TestNewConnWhileRunning() {
connMock := &testlib.EssentialsConnMock{}
connMock.
On("Write", mock.AnythingOfType("[]uint8")).
Return(0, nil).
Maybe()
connMock.On("Close").
Return(nil).
Maybe()
conn, err := suite.g.NewConn(connMock)
suite.NoError(err)
conn.Stop()
}
func (suite *GangerTestSuite) TestNewConnWriteProducesTLSRecords() {
var (
mu sync.Mutex
buf bytes.Buffer
)
connMock := &testlib.EssentialsConnMock{}
connMock.On("Write", mock.AnythingOfType("[]uint8")).
Run(func(args mock.Arguments) {
mu.Lock()
buf.Write(args.Get(0).([]byte))
mu.Unlock()
}).
Return(0, nil).
Maybe()
connMock.On("Close").
Return(nil).
Maybe()
conn, err := suite.g.NewConn(connMock)
suite.NoError(err)
payload := bytes.Repeat([]byte("x"), 512)
_, err = conn.Write(payload)
suite.NoError(err)
time.Sleep(500 * time.Millisecond)
conn.Stop()
mu.Lock()
written := buf.Bytes()
mu.Unlock()
suite.NotEmpty(written)
}
func TestGanger(t *testing.T) {
t.Parallel()
suite.Run(t, &GangerTestSuite{})
}
+43
View File
@@ -0,0 +1,43 @@
package doppel
import (
"context"
"net"
"net/http"
"time"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
const (
// Please see Stats description
// https://blog.cloudflare.com/optimizing-tls-over-tcp-to-reduce-latency/
// https://github.com/cloudflare/sslconfig/blob/master/patches/nginx__dynamic_tls_records.patch
TLSRecordSizeStart = 1450
TLSRecordSizeAccel = 4096
TLSRecordSizeMax = 16384 - tls.SizeHeader
TLSCounterAccelAfter = 40
TLSCounterMaxAfter = TLSCounterAccelAfter + 20
TLSRecordSizeResetAfter = time.Second
)
// copypasted from mtglib
type Network interface {
// Dial establishes context-free TCP connections.
Dial(network, address string) (essentials.Conn, error)
// DialContext dials using a context. This is a preferable way of
// establishing TCP connections.
DialContext(ctx context.Context, network, address string) (essentials.Conn, error)
// MakeHTTPClient build an HTTP client with given dial function. If nothing is
// provided, then DialContext of this interface is going to be used.
MakeHTTPClient(func(ctx context.Context, network, address string) (essentials.Conn, error)) *http.Client
// NativeDialer returns a configured instance of native dialer that
// skips proxy connections or any other irrelevant settings.
NativeDialer() *net.Dialer
}
+107
View File
@@ -0,0 +1,107 @@
package doppel
import (
"context"
"crypto/tls"
"net"
"net/http"
"net/http/httptest"
"time"
"github.com/9seconds/mtg/v2/essentials"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/suite"
)
type SimpleNetwork struct{}
func (s SimpleNetwork) Dial(network, address string) (essentials.Conn, error) {
return s.DialContext(context.Background(), network, address)
}
func (s SimpleNetwork) DialContext(ctx context.Context, network, address string) (essentials.Conn, error) {
d := &net.Dialer{}
conn, err := d.DialContext(ctx, network, address)
if err != nil {
return nil, err
}
return conn.(*net.TCPConn), nil
}
func (s SimpleNetwork) NativeDialer() *net.Dialer {
return &net.Dialer{}
}
func (s SimpleNetwork) MakeHTTPClient(dialFunc func(ctx context.Context, network, address string) (essentials.Conn, error)) *http.Client {
if dialFunc == nil {
dialFunc = s.DialContext
}
return &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true, //nolint: gosec
},
DialContext: func(ctx context.Context, network, address string) (net.Conn, error) {
return dialFunc(ctx, network, address)
},
},
}
}
type TLSServerTestSuite struct {
suite.Suite
tlsServer *httptest.Server
ctx context.Context
ctxCancel context.CancelFunc
network SimpleNetwork
urls []string
}
func (suite *TLSServerTestSuite) SetupSuite() {
suite.tlsServer = httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
w.WriteHeader(http.StatusOK)
w.Header().Add("Hello", "how long")
if _, err := w.Write([]byte{1, 2, 3}); err != nil {
panic(err)
}
time.Sleep(5 * time.Millisecond)
if _, err := w.Write([]byte{1, 2, 3}); err != nil {
panic(err)
}
}))
suite.urls = []string{suite.tlsServer.URL}
}
func (suite *TLSServerTestSuite) SetupTest() {
ctx, cancel := context.WithCancel(context.Background())
suite.ctx = ctx
suite.ctxCancel = cancel
}
func (suite *TLSServerTestSuite) TearDownTest() {
suite.ctxCancel()
suite.tlsServer.CloseClientConnections()
}
func (suite *TLSServerTestSuite) TearDownSuite() {
suite.tlsServer.Close()
}
type LoggerMock struct {
mock.Mock
}
func (l *LoggerMock) Info(msg string) {
l.Called(msg)
}
func (l *LoggerMock) WarningError(msg string, err error) {
l.Called(msg, err)
}
+6
View File
@@ -0,0 +1,6 @@
package doppel
type Logger interface {
Info(msg string)
WarningError(msg string, err error)
}
+146
View File
@@ -0,0 +1,146 @@
package doppel
import (
"context"
"fmt"
"io"
"net/http"
"strings"
"time"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
// ScoutResult holds measurements from a single scout HTTP request.
type ScoutResult struct {
Durations []time.Duration
CertSize int // total ApplicationData bytes during TLS handshake; 0 if unknown
}
type Scout struct {
network Network
urls []string
}
func (s Scout) Learn(ctx context.Context) (ScoutResult, error) {
var combined ScoutResult
for _, url := range s.urls {
learned, err := s.learn(ctx, url)
if err != nil {
return ScoutResult{}, err
}
combined.Durations = append(combined.Durations, learned.Durations...)
if learned.CertSize > 0 && combined.CertSize == 0 {
combined.CertSize = learned.CertSize
}
}
return combined, nil
}
func (s Scout) learn(ctx context.Context, url string) (ScoutResult, error) {
client, results := s.makeClient()
if !strings.HasPrefix(url, "https://") {
return ScoutResult{}, fmt.Errorf("url %s must be https", url)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return ScoutResult{}, err
}
resp, err := client.Do(req)
if resp != nil {
io.Copy(io.Discard, resp.Body) //nolint: errcheck
resp.Body.Close() //nolint: errcheck
client.CloseIdleConnections()
}
if err != nil {
return ScoutResult{}, err
}
data, writeIndex := results.Snapshot()
if len(data) == 0 {
return ScoutResult{}, nil
}
var result ScoutResult
// Compute inter-record durations (existing logic).
lastTimestamp := time.Time{}
for i, v := range data {
if v.recordType != tls.TypeApplicationData {
continue
}
if lastTimestamp.IsZero() {
if i > 0 {
lastTimestamp = data[i-1].timestamp
} else {
lastTimestamp = v.timestamp
}
}
result.Durations = append(result.Durations, v.timestamp.Sub(lastTimestamp))
lastTimestamp = v.timestamp
}
// Compute cert size: sum of ApplicationData payload between CCS and
// the first client Write (which marks the end of server handshake).
seenCCS := false
boundary := writeIndex
if boundary < 0 {
boundary = len(data)
}
for i, v := range data {
if i >= boundary {
break
}
if v.recordType == tls.TypeChangeCipherSpec {
seenCCS = true
continue
}
if seenCCS && v.recordType == tls.TypeApplicationData {
result.CertSize += v.payloadLen
}
}
return result, nil
}
func (s Scout) makeClient() (*http.Client, *ScoutConnCollected) {
dialer := s.network.NativeDialer()
collected := NewScoutConnCollected()
client := s.network.MakeHTTPClient(func(
ctx context.Context,
network string,
address string,
) (essentials.Conn, error) {
conn, err := dialer.DialContext(ctx, network, address)
if err != nil {
return nil, err
}
return NewScoutConn(essentials.WrapNetConn(conn), collected), nil
})
return client, collected
}
func NewScout(network Network, urls []string) Scout {
return Scout{
network: network,
urls: urls,
}
}
+70
View File
@@ -0,0 +1,70 @@
package doppel
import (
"bytes"
"encoding/binary"
"io"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
)
type ScoutConn struct {
tls.Conn
results *ScoutConnCollected
rawBuf *bytes.Buffer
seenCCS bool
}
func (s *ScoutConn) Read(p []byte) (int, error) {
buf := &bytes.Buffer{}
for {
if n, err := s.rawBuf.Read(p); err == nil {
return n, nil
}
s.rawBuf.Reset()
recordType, length, err := tls.ReadRecord(s.Conn, buf)
if err != nil {
return 0, err
}
if recordType == tls.TypeChangeCipherSpec {
s.seenCCS = true
}
s.results.Add(recordType, int(length))
s.rawBuf.Write([]byte{recordType})
s.rawBuf.Write(tls.TLSVersion[:])
if err := binary.Write(s.rawBuf, binary.BigEndian, uint16(length)); err != nil {
return 0, err
}
if _, err := io.Copy(s.rawBuf, buf); err != nil {
return 0, err
}
}
}
func (s *ScoutConn) Write(p []byte) (int, error) {
if s.seenCCS {
s.results.MarkWrite()
}
return s.Conn.Write(p)
}
func NewScoutConn(conn essentials.Conn, results *ScoutConnCollected) *ScoutConn {
rawBuf := &bytes.Buffer{}
rawBuf.Grow(tls.MaxRecordSize)
return &ScoutConn{
Conn: tls.New(conn, false, false),
results: results,
rawBuf: rawBuf,
}
}
@@ -0,0 +1,59 @@
package doppel
import (
"slices"
"sync"
"time"
)
const (
ScoutConnCollectedPreallocSize = 100
)
type ScoutConnResult struct {
timestamp time.Time
recordType byte
payloadLen int
}
type ScoutConnCollected struct {
mu sync.Mutex
data []ScoutConnResult
writeIndex int // index at which client first wrote post-handshake data; -1 if not set
}
func (s *ScoutConnCollected) Add(record byte, payloadLen int) {
s.mu.Lock()
s.data = append(s.data, ScoutConnResult{
timestamp: time.Now(),
recordType: record,
payloadLen: payloadLen,
})
s.mu.Unlock()
}
// MarkWrite records the current data length as the handshake boundary.
func (s *ScoutConnCollected) MarkWrite() {
s.mu.Lock()
if s.writeIndex < 0 {
s.writeIndex = len(s.data)
}
s.mu.Unlock()
}
// Snapshot returns a copy of the collected data and the write index.
func (s *ScoutConnCollected) Snapshot() ([]ScoutConnResult, int) {
s.mu.Lock()
snapshot := slices.Clone(s.data)
writeIndex := s.writeIndex
s.mu.Unlock()
return snapshot, writeIndex
}
func NewScoutConnCollected() *ScoutConnCollected {
return &ScoutConnCollected{
data: make([]ScoutConnResult, 0, ScoutConnCollectedPreallocSize),
writeIndex: -1,
}
}
@@ -0,0 +1,86 @@
package doppel
import (
"sync"
"testing"
"time"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
"github.com/stretchr/testify/suite"
)
type ScoutConnCollectedTestSuite struct {
suite.Suite
}
func (suite *ScoutConnCollectedTestSuite) TestAddSingle() {
collected := NewScoutConnCollected()
collected.Add(tls.TypeApplicationData, 100)
data, _ := collected.Snapshot()
suite.Len(data, 1)
suite.Equal(byte(tls.TypeApplicationData), data[0].recordType)
}
func (suite *ScoutConnCollectedTestSuite) TestAddTimestampsAreMonotonic() {
collected := NewScoutConnCollected()
collected.Add(tls.TypeApplicationData, 100)
time.Sleep(time.Microsecond)
collected.Add(tls.TypeApplicationData, 100)
time.Sleep(time.Microsecond)
collected.Add(tls.TypeApplicationData, 100)
data, _ := collected.Snapshot()
for i := 1; i < len(data); i++ {
suite.True(data[i].timestamp.After(data[i-1].timestamp))
}
}
func (suite *ScoutConnCollectedTestSuite) TestConcurrentAddSnapshot() {
collected := NewScoutConnCollected()
var wg sync.WaitGroup
wg.Add(3)
go func() {
defer wg.Done()
for i := 0; i < 1000; i++ {
collected.Add(tls.TypeApplicationData, i)
}
}()
go func() {
defer wg.Done()
for i := 0; i < 100; i++ {
collected.MarkWrite()
}
}()
go func() {
defer wg.Done()
for i := 0; i < 1000; i++ {
// call Snapshot concurrently to exercise the lock under -race
collected.Snapshot() //nolint:errcheck
}
}()
wg.Wait()
data, writeIndex := collected.Snapshot()
suite.Len(data, 1000)
suite.GreaterOrEqual(writeIndex, 0)
}
func TestScoutConnCollected(t *testing.T) {
t.Parallel()
suite.Run(t, &ScoutConnCollectedTestSuite{})
}
+39
View File
@@ -0,0 +1,39 @@
package doppel
import (
"testing"
"github.com/stretchr/testify/suite"
)
type ScoutTestSuite struct {
TLSServerTestSuite
scout Scout
}
func (suite *ScoutTestSuite) SetupSuite() {
suite.TLSServerTestSuite.SetupSuite()
suite.scout = Scout{
network: suite.network,
urls: suite.urls,
}
}
func (suite *ScoutTestSuite) TestCollectResults() {
result, err := suite.scout.Learn(suite.ctx)
suite.NoError(err)
suite.Less(3, len(result.Durations))
}
func (suite *ScoutTestSuite) TestCollectNothing() {
suite.ctxCancel()
_, err := suite.scout.Learn(suite.ctx)
suite.Error(err)
}
func TestScout(t *testing.T) {
suite.Run(t, &ScoutTestSuite{})
}
+170
View File
@@ -0,0 +1,170 @@
package doppel
import (
"math"
"math/rand/v2"
"time"
)
const (
StatsBisectTimes = 70
StatsLowK = 0.01
StatsHighK = 10.0
// do not calculate statistics if we have < than this number of durations
MinDurationsToCalculate = 100
// these values are taken from ok.ru. measured from moscow site.
StatsDefaultK = 0.37846373895785335
StatsDefaultLambda = 1.73177086015485
// how many bytes should we drift
DRSNoise = 100
)
// Stats is responsible for generating values that are distributed according
// to some statistical distribution.
//
// It follows several ideas:
// 1. Based on nginx and Cloudflare behaviour, even if server is eager
// to send a lot, they all start with small TLS packets that are
// approximately MTU-sized. After
// 2. After ~40 TLS records, server considers TCP session as somewhat solid
// and reliable and ramps up to 4096.
// 3. After ~20 TLS records more it jumps to the max 16384 bytes and keep
// this size as long as it can
// 4. If there is no any byte within a connection for a longer time period,
// this counter resets.
//
// This is called Dynamic TLS Record Sizing
// - https://blog.cloudflare.com/optimizing-tls-over-tcp-to-reduce-latency/
// - https://community.f5.com/kb/technicalarticles/boosting-tls-performance-with-dynamic-record-sizing-on-big-ip/280798
// - https://www.igvita.com/2013/10/24/optimizing-tls-record-size-and-buffering-latency/
//
// And this optimized for the very first byte, so web browsers could start to
// render as early as possible, showing user some preliminary results, optimizing
// for perceived latency.
//
// Since this is very typical for the website, we also aim for that.
//
// Another important idea is how delays between TLS packets are distributed.
// In case of sending huge heavy content with max sized record, delays have
// lognormal distribution. But a nature of a typical website shows that
// it eagers to deliver as fast as it can in a few very first records and
// could possibly slow down later.
//
// This is perfectly described by Weibull distribution:
// - https://en.wikipedia.org/wiki/Weibull_distribution
// - https://ieeexplore.ieee.org/document/6662948
// - https://www.researchgate.net/publication/224621285_Traffic_modelling_and_cost_optimization_for_transmitting_traffic_messages_over_a_hybrid_broadcast_and_cellular_network
// - https://ir.uitm.edu.my/id/eprint/105386/1/105386.pdf
//
// In other word, a combination of Dynamic TLS Record Sizing hints us for
// Weibull distribution.
//
// But we also have to keep in mind that DRS is not well spread yet. In most cases
// users still rely on OpenSSL or webserver defaults. OpenSSL chunks with
// biggest packet sizes, nginx relies on static setting that is 16k by default.
// Thus, dynamic sizing has to be present but we cannot oblige users to use that.
type Stats struct {
sizeLastRequested time.Time
sizeCounter int
// https://en.wikipedia.org/wiki/Shape_parameter
k float64
// https://en.wikipedia.org/wiki/Scale_parameter
lambda float64
// Dynamic Record Sizing
drs bool
}
func (d *Stats) Delay() time.Duration {
// u ∈ (0, 1], avoids ln(0)
u := 1.0 - rand.Float64()
// X = λ·(-ln U)^(1/k)
generated := d.lambda * math.Pow(-math.Log(u), 1.0/d.k)
// generated is in milliseconds
return time.Duration(generated * float64(time.Millisecond))
}
func (d *Stats) Size() int {
if time.Since(d.sizeLastRequested) > TLSRecordSizeResetAfter {
d.sizeCounter = 0
}
if !d.drs {
return TLSRecordSizeMax
}
d.sizeLastRequested = time.Now()
d.sizeCounter++
switch {
case d.sizeCounter <= TLSCounterAccelAfter:
return TLSRecordSizeStart - rand.IntN(DRSNoise)
case d.sizeCounter <= TLSCounterMaxAfter:
return TLSRecordSizeAccel - rand.IntN(DRSNoise)
}
return TLSRecordSizeMax
}
func NewStats(durations []time.Duration, drs bool) Stats {
n := float64(len(durations))
// in milliseconds
durFloats := make([]float64, len(durations))
for i, v := range durations {
durFloats[i] = float64(v.Microseconds()) / 1000.0
}
// The bisection solves the standard Weibull MLE equation for shape
// parameter k. There is no any good formula for doing that so we
// approximate it by several bisections. The number of operations
// is statically defined by a constant.
sumLog := 0.0
for _, v := range durFloats {
sumLog += math.Log(v)
}
lowK := StatsLowK
highK := StatsHighK
for range StatsBisectTimes {
midK := (lowK + highK) / 2.0
sumXK := 0.0
sumXKLog := 0.0
for _, v := range durFloats {
xk := math.Pow(v, midK)
sumXK += xk
sumXKLog += xk * math.Log(v)
}
if (1.0/midK)+(sumLog/n)-(sumXKLog/sumXK) > 0 {
lowK = midK
} else {
highK = midK
}
}
k := (lowK + highK) / 2
sumXK := 0.0
for _, v := range durFloats {
sumXK += math.Pow(v, k)
}
// λ = (Σxᵢᵏ / n)^(1/k)
lambda := math.Pow(sumXK/n, 1.0/k)
return Stats{
k: k,
lambda: lambda,
drs: drs,
}
}
+219
View File
@@ -0,0 +1,219 @@
package doppel
import (
"math"
"math/rand/v2"
"testing"
"time"
"github.com/stretchr/testify/suite"
)
type StatsTestSuite struct {
suite.Suite
}
func (suite *StatsTestSuite) GenWeibull(k, lambda float64, n int, seed uint64) []time.Duration {
rng := rand.New(rand.NewPCG(seed, 0))
samples := make([]time.Duration, n)
for i := range samples {
u := 1.0 - rng.Float64()
ms := lambda * math.Pow(-math.Log(u), 1.0/k)
d := time.Duration(ms * float64(time.Millisecond))
if d < time.Microsecond {
time.Sleep(time.Microsecond)
d = time.Microsecond
}
samples[i] = d
}
return samples
}
func (suite *StatsTestSuite) TestNewStatsRecoverParameters() {
knownK := 1.5
knownLambda := 100.0
samples := suite.GenWeibull(knownK, knownLambda, 5000, 42)
stats := NewStats(samples, true)
suite.InDelta(knownK, stats.k, 0.1)
suite.InDelta(knownLambda, stats.lambda, 5.0)
}
func (suite *StatsTestSuite) TestNewStatsExponentialCase() {
// When k=1, Weibull reduces to exponential distribution.
knownK := 1.0
knownLambda := 50.0
samples := suite.GenWeibull(knownK, knownLambda, 5000, 123)
stats := NewStats(samples, true)
suite.InDelta(knownK, stats.k, 0.1)
suite.InDelta(knownLambda, stats.lambda, 5.0)
}
func (suite *StatsTestSuite) TestNewStatsSmallK() {
// k < 1 produces a heavy-tailed distribution typical for network delays.
// Lambda must be large enough so samples stay above microsecond precision
// after time.Duration round-trip.
knownK := 0.6
knownLambda := 100.0
samples := suite.GenWeibull(knownK, knownLambda, 10000, 99)
stats := NewStats(samples, true)
suite.InDelta(knownK, stats.k, 0.05)
suite.InDelta(knownLambda, stats.lambda, 5.0)
}
func (suite *StatsTestSuite) TestNewStatsLargeK() {
// k > 1: light tail, concentrated around the mode.
knownK := 5.0
knownLambda := 200.0
samples := suite.GenWeibull(knownK, knownLambda, 5000, 77)
stats := NewStats(samples, true)
suite.InDelta(knownK, stats.k, 0.3)
suite.InDelta(knownLambda, stats.lambda, 5.0)
}
func (suite *StatsTestSuite) TestDelayNonNegative() {
stats := &Stats{
k: 1.5,
lambda: 100.0,
}
for range 200 {
dur := stats.Delay()
suite.GreaterOrEqual(dur, time.Duration(0))
}
}
func (suite *StatsTestSuite) TestDelayDistributionMean() {
// Weibull mean = λ · Γ(1 + 1/k)
k := 2.0
lambda := 50.0
stats := &Stats{k: k, lambda: lambda}
n := 50000
sum := 0.0
for range n {
dur := stats.Delay()
sum += float64(dur) / float64(time.Millisecond)
}
sampleMean := sum / float64(n)
expectedMean := lambda * math.Gamma(1.0+1.0/k)
suite.InDelta(expectedMean, sampleMean, expectedMean*0.05)
}
func (suite *StatsTestSuite) TestNewStatsRoundTrip() {
// Estimate parameters from data, then verify that Delay samples
// from the fitted distribution have approximately the same mean.
knownK := 1.2
knownLambda := 80.0
samples := suite.GenWeibull(knownK, knownLambda, 5000, 555)
stats := NewStats(samples, true)
n := 50000
sum := 0.0
for range n {
dur := stats.Delay()
sum += float64(dur) / float64(time.Millisecond)
}
sampleMean := sum / float64(n)
expectedMean := knownLambda * math.Gamma(1.0+1.0/knownK)
suite.InDelta(expectedMean, sampleMean, expectedMean*0.05)
}
func (suite *StatsTestSuite) TestSizeStartPhase() {
stats := &Stats{k: 1.0, lambda: 1.0, drs: true}
for range TLSCounterAccelAfter {
size := stats.Size()
suite.GreaterOrEqual(size, TLSRecordSizeStart-DRSNoise)
suite.LessOrEqual(size, TLSRecordSizeStart)
}
}
func (suite *StatsTestSuite) TestSizeAccelPhase() {
stats := &Stats{k: 1.0, lambda: 1.0, drs: true}
for range TLSCounterAccelAfter {
stats.Size()
}
for range TLSCounterMaxAfter - TLSCounterAccelAfter {
size := stats.Size()
suite.GreaterOrEqual(size, TLSRecordSizeAccel-DRSNoise)
suite.LessOrEqual(size, TLSRecordSizeAccel)
}
}
func (suite *StatsTestSuite) TestSizeMaxPhase() {
stats := &Stats{k: 1.0, lambda: 1.0, drs: true}
for range TLSCounterMaxAfter {
stats.Size()
}
for range 20 {
size := stats.Size()
suite.Equal(TLSRecordSizeMax, size)
}
}
func (suite *StatsTestSuite) TestSizeResetsAfterInactivity() {
stats := &Stats{k: 1.0, lambda: 1.0, drs: true}
// Advance past start phase.
for range TLSCounterMaxAfter {
stats.Size()
}
suite.Equal(TLSRecordSizeMax, stats.Size())
// Simulate inactivity by backdating sizeLastRequested.
stats.sizeLastRequested = time.Now().Add(-TLSRecordSizeResetAfter - time.Millisecond)
size := stats.Size()
suite.GreaterOrEqual(size, TLSRecordSizeStart-DRSNoise)
suite.LessOrEqual(size, TLSRecordSizeStart)
}
func (suite *StatsTestSuite) TestSizeNoDRSAlwaysMax() {
stats := &Stats{k: 1.0, lambda: 1.0, drs: false}
for range TLSCounterMaxAfter + 20 {
suite.Equal(TLSRecordSizeMax, stats.Size())
}
}
func (suite *StatsTestSuite) TestSizeNoDRSIgnoresCounter() {
stats := &Stats{k: 1.0, lambda: 1.0, drs: false}
// Even after many calls, always returns max.
for range 200 {
suite.Equal(TLSRecordSizeMax, stats.Size())
}
// Inactivity has no effect either.
stats.sizeLastRequested = time.Now().Add(-TLSRecordSizeResetAfter - time.Millisecond)
suite.Equal(TLSRecordSizeMax, stats.Size())
}
func TestStats(t *testing.T) {
t.Parallel()
suite.Run(t, &StatsTestSuite{})
}
-134
View File
@@ -1,134 +0,0 @@
package faketls
import (
"crypto/hmac"
"crypto/sha256"
"crypto/subtle"
"encoding/binary"
"fmt"
"time"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls/record"
)
type ClientHello struct {
Time time.Time
Random [RandomLen]byte
SessionID []byte
Host string
CipherSuite uint16
}
func (c ClientHello) Valid(hostname string, tolerateTimeSkewness time.Duration) error {
if c.Host != "" && c.Host != hostname {
return fmt.Errorf("incorrect hostname %s", hostname)
}
now := time.Now()
timeDiff := now.Sub(c.Time)
if timeDiff < 0 {
timeDiff = -timeDiff
}
if timeDiff > tolerateTimeSkewness {
return fmt.Errorf("incorrect timestamp. got=%d, now=%d, diff=%s",
c.Time.Unix(), now.Unix(), timeDiff.String())
}
return nil
}
func ParseClientHello(secret, handshake []byte) (ClientHello, error) {
hello := ClientHello{}
if len(handshake) < ClientHelloMinLen {
return hello, fmt.Errorf("lengh of handshake is too small: %d", len(handshake))
}
if handshake[0] != HandshakeTypeClient {
return hello, fmt.Errorf("unknown handshake type %#x", handshake[0])
}
handshakeSizeBytes := [4]byte{0, handshake[1], handshake[2], handshake[3]}
handshakeLength := binary.BigEndian.Uint32(handshakeSizeBytes[:])
if len(handshake)-4 != int(handshakeLength) {
return hello,
fmt.Errorf("incorrect handshake size. manifested=%d, real=%d",
handshakeLength, len(handshake)-4)
}
copy(hello.Random[:], handshake[ClientHelloRandomOffset:])
copy(handshake[ClientHelloRandomOffset:], clientHelloEmptyRandom)
rec := record.AcquireRecord()
defer record.ReleaseRecord(rec)
rec.Type = record.TypeHandshake
rec.Version = record.Version10
rec.Payload.Write(handshake)
// mac is calculated for the whole record, not only
// for the payload part
mac := hmac.New(sha256.New, secret)
rec.Dump(mac) //nolint: errcheck
computedRandom := mac.Sum(nil)
for i := range RandomLen {
computedRandom[i] ^= hello.Random[i]
}
if subtle.ConstantTimeCompare(clientHelloEmptyRandom[:RandomLen-4], computedRandom[:RandomLen-4]) != 1 {
return hello, ErrBadDigest
}
timestamp := int64(binary.LittleEndian.Uint32(computedRandom[RandomLen-4:]))
hello.Time = time.Unix(timestamp, 0)
parseSessionID(&hello, handshake)
parseCipherSuite(&hello, handshake)
parseSNI(&hello, handshake)
return hello, nil
}
func parseSessionID(hello *ClientHello, handshake []byte) {
hello.SessionID = make([]byte, handshake[ClientHelloSessionIDOffset])
copy(hello.SessionID, handshake[ClientHelloSessionIDOffset+1:])
}
func parseCipherSuite(hello *ClientHello, handshake []byte) {
cipherSuiteOffset := ClientHelloSessionIDOffset + len(hello.SessionID) + 3
hello.CipherSuite = binary.BigEndian.Uint16(handshake[cipherSuiteOffset : cipherSuiteOffset+2])
}
func parseSNI(hello *ClientHello, handshake []byte) {
cipherSuiteOffset := ClientHelloSessionIDOffset + len(hello.SessionID) + 1
handshake = handshake[cipherSuiteOffset:]
cipherSuiteLength := binary.BigEndian.Uint16(handshake[:2])
handshake = handshake[2+cipherSuiteLength:]
compressionMethodsLength := int(handshake[0])
handshake = handshake[1+compressionMethodsLength:]
extensionsLength := binary.BigEndian.Uint16(handshake[:2])
handshake = handshake[2 : 2+extensionsLength]
for len(handshake) > 0 {
if binary.BigEndian.Uint16(handshake[:2]) != ExtensionSNI {
extensionsLength := binary.BigEndian.Uint16(handshake[2:4])
handshake = handshake[4+extensionsLength:]
continue
}
hostnameLength := binary.BigEndian.Uint16(handshake[7:9])
handshake = handshake[9:]
hello.Host = string(handshake[:int(hostnameLength)])
return
}
}
@@ -1,21 +0,0 @@
package faketls_test
import (
"testing"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls"
"github.com/stretchr/testify/require"
)
var FuzzClientHelloSecret = []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
func FuzzClientHello(f *testing.F) {
f.Add([]byte{1, 2, 3})
f.Fuzz(func(t *testing.T, frame []byte) {
_, err := faketls.ParseClientHello(FuzzClientHelloSecret, frame)
// a probability of having != err is almost negligible
require.Error(t, err)
})
}
@@ -1,191 +0,0 @@
package faketls_test
import (
"encoding/base64"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/suite"
)
type ClientHelloSnapshot struct {
Time int `json:"time"`
Random string `json:"random"`
SessionID string `json:"sessionId"`
Host string `json:"host"`
CipherSuite int `json:"cipherSuite"`
Full string `json:"full"`
}
func (c ClientHelloSnapshot) GetTime() time.Time {
return time.Unix(int64(c.Time), 0)
}
func (c ClientHelloSnapshot) GetRandom() []byte {
data, _ := base64.StdEncoding.DecodeString(c.Random)
return data
}
func (c ClientHelloSnapshot) GetSessionID() []byte {
data, _ := base64.StdEncoding.DecodeString(c.SessionID)
return data
}
func (c ClientHelloSnapshot) GetHost() string {
return c.Host
}
func (c ClientHelloSnapshot) GetCipherSuite() uint16 {
return uint16(c.CipherSuite)
}
func (c ClientHelloSnapshot) GetFull() []byte {
data, _ := base64.StdEncoding.DecodeString(c.Full)
return data
}
type ClientHelloTestSuite struct {
suite.Suite
secret mtglib.Secret
}
func (suite *ClientHelloTestSuite) SetupSuite() {
parsed, err := mtglib.ParseSecret("ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d")
if err != nil {
panic(err)
}
suite.secret = parsed
}
func (suite *ClientHelloTestSuite) TestEmptyHandshake() {
_, err := faketls.ParseClientHello(suite.secret.Key[:], nil)
suite.Error(err)
}
func (suite *ClientHelloTestSuite) TestIncorrectHandshakeType() {
data := make([]byte, 1024)
data[0] = 0x02
_, err := faketls.ParseClientHello(suite.secret.Key[:], data)
suite.Error(err)
}
func (suite *ClientHelloTestSuite) TestIncorrectLength() {
data := make([]byte, 1024)
data[0] = 0x01
data[1] = 0xff
data[2] = 0xff
_, err := faketls.ParseClientHello(suite.secret.Key[:], data)
suite.Error(err)
}
func (suite *ClientHelloTestSuite) TestSnapshotOk() {
files, err := os.ReadDir("testdata")
suite.NoError(err)
testData := []string{}
for _, v := range files {
if strings.HasPrefix(v.Name(), "client-hello-ok") {
testData = append(testData, v.Name())
}
}
for _, name := range testData {
path := filepath.Join("testdata", name)
suite.T().Run(name, func(t *testing.T) {
fileData, err := os.ReadFile(path)
assert.NoError(t, err)
snapshot := &ClientHelloSnapshot{}
assert.NoError(t, json.Unmarshal(fileData, snapshot))
hello, err := faketls.ParseClientHello(suite.secret.Key[:], snapshot.GetFull())
assert.NoError(t, err)
assert.WithinDuration(t, snapshot.GetTime(), hello.Time, time.Second)
assert.Equal(t, snapshot.GetRandom(), hello.Random[:])
assert.Equal(t, snapshot.GetSessionID(), hello.SessionID)
assert.Equal(t, snapshot.GetHost(), hello.Host)
assert.Equal(t, snapshot.GetCipherSuite(), hello.CipherSuite)
})
}
}
func (suite *ClientHelloTestSuite) TestSnapshotBad() {
files, err := os.ReadDir("testdata")
suite.NoError(err)
testData := []string{}
for _, v := range files {
if strings.HasPrefix(v.Name(), "client-hello-bad") {
testData = append(testData, v.Name())
}
}
for _, name := range testData {
path := filepath.Join("testdata", name)
suite.T().Run(name, func(t *testing.T) {
fileData, err := os.ReadFile(path)
assert.NoError(t, err)
snapshot := &ClientHelloSnapshot{}
assert.NoError(t, json.Unmarshal(fileData, snapshot))
_, err = faketls.ParseClientHello(suite.secret.Key[:], snapshot.GetFull())
assert.Error(t, err)
})
}
}
func (suite *ClientHelloTestSuite) TestValidateHostname() {
hello := faketls.ClientHello{
Time: time.Now(),
}
suite.NoError(hello.Valid("hostname", time.Second))
hello.Host = "hostname"
suite.Error(hello.Valid("hostname2", time.Second))
suite.NoError(hello.Valid("hostname", time.Second))
}
func (suite *ClientHelloTestSuite) TestValidateTime() {
testData := []time.Duration{
-2 * time.Second,
2 * time.Second,
}
for _, v := range testData {
value := v
suite.T().Run(value.String(), func(t *testing.T) {
hello := faketls.ClientHello{
Host: "hostname",
Time: time.Now().Add(value),
}
suite.Error(hello.Valid("hostname", 500*time.Millisecond))
suite.Error(hello.Valid("hostname", time.Second))
suite.NoError(hello.Valid("hostname", 3*time.Second))
})
}
}
func TestClientHello(t *testing.T) {
t.Parallel()
suite.Run(t, &ClientHelloTestSuite{})
}
-72
View File
@@ -1,72 +0,0 @@
package faketls
import (
"bytes"
"fmt"
"math/rand/v2"
"github.com/9seconds/mtg/v2/essentials"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls/record"
)
type Conn struct {
essentials.Conn
readBuffer bytes.Buffer
}
func (c *Conn) Read(p []byte) (int, error) {
if n, _ := c.readBuffer.Read(p); n > 0 {
return n, nil
}
rec := record.AcquireRecord()
defer record.ReleaseRecord(rec)
for {
if err := rec.Read(c.Conn); err != nil {
return 0, err //nolint: wrapcheck
}
switch rec.Type { //nolint: exhaustive
case record.TypeApplicationData:
rec.Payload.WriteTo(&c.readBuffer) //nolint: errcheck
return c.readBuffer.Read(p) //nolint: wrapcheck
case record.TypeChangeCipherSpec:
default:
return 0, fmt.Errorf("unsupported record type %v", rec.Type)
}
}
}
func (c *Conn) Write(p []byte) (int, error) {
rec := record.AcquireRecord()
defer record.ReleaseRecord(rec)
rec.Type = record.TypeApplicationData
rec.Version = record.Version12
written := 0
for len(p) > 0 {
chunkSize := rand.IntN(record.TLSMaxRecordSize)
if chunkSize > len(p) || chunkSize == 0 {
chunkSize = len(p)
}
rec.Payload.Reset()
rec.Payload.Write(p[:chunkSize])
err := rec.Dump(c.Conn)
written += chunkSize
if err != nil {
return written, err
}
p = p[chunkSize:]
}
return written, nil
}
-153
View File
@@ -1,153 +0,0 @@
package faketls_test
import (
"bytes"
"crypto/rand"
"errors"
"io"
"testing"
"github.com/9seconds/mtg/v2/internal/testlib"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls/record"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/suite"
)
type ConnMock struct {
testlib.EssentialsConnMock
readBuffer bytes.Buffer
writeBuffer bytes.Buffer
}
func (m *ConnMock) Read(p []byte) (int, error) {
m.Called(p)
return m.readBuffer.Read(p) //nolint: wrapcheck
}
func (m *ConnMock) Write(p []byte) (int, error) {
m.Called(p)
return m.writeBuffer.Write(p) //nolint: wrapcheck
}
type ConnTestSuite struct {
suite.Suite
connMock *ConnMock
c *faketls.Conn
}
func (suite *ConnTestSuite) SetupTest() {
suite.connMock = &ConnMock{}
suite.c = &faketls.Conn{
Conn: suite.connMock,
}
}
func (suite *ConnTestSuite) TearDownTest() {
suite.connMock.AssertExpectations(suite.T())
}
func (suite *ConnTestSuite) TestRead() {
suite.connMock.On("Read", mock.Anything).Return(0, nil)
rec := record.AcquireRecord()
defer record.ReleaseRecord(rec)
rec.Type = record.TypeChangeCipherSpec
rec.Version = record.Version12
rec.Payload.WriteByte(0x01)
rec.Dump(&suite.connMock.readBuffer) //nolint: errcheck
rec.Reset()
rec.Type = record.TypeApplicationData
rec.Version = record.Version12
rec.Payload.Write([]byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10})
rec.Dump(&suite.connMock.readBuffer) //nolint: errcheck
resultBuffer := &bytes.Buffer{}
buf := make([]byte, 2)
for {
n, err := suite.c.Read(buf)
if errors.Is(err, io.EOF) {
break
}
resultBuffer.Write(buf[:n])
}
suite.Equal([]byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, resultBuffer.Bytes())
}
func (suite *ConnTestSuite) TestReadUnexpected() {
suite.connMock.On("Read", mock.Anything).Return(0, nil)
rec := record.AcquireRecord()
defer record.ReleaseRecord(rec)
rec.Type = record.TypeChangeCipherSpec
rec.Version = record.Version12
rec.Payload.WriteByte(0x01)
rec.Dump(&suite.connMock.readBuffer) //nolint: errcheck
rec.Reset()
rec.Type = record.TypeHandshake
rec.Version = record.Version12
rec.Payload.Write([]byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10})
rec.Dump(&suite.connMock.readBuffer) //nolint: errcheck
buf := make([]byte, 2)
for {
_, err := suite.c.Read(buf)
switch {
case err == nil:
case errors.Is(err, io.EOF):
suite.FailNow("unexpected to finish")
default:
return
}
}
}
func (suite *ConnTestSuite) TestWrite() {
suite.connMock.On("Write", mock.Anything).Return(0, nil)
dataToRec := make([]byte, record.TLSMaxRecordSize*2)
rand.Read(dataToRec) //nolint: staticcheck, errcheck
n, err := suite.c.Write(dataToRec)
suite.NoError(err)
suite.Equal(len(dataToRec), n)
rec := record.AcquireRecord()
defer record.ReleaseRecord(rec)
buf := &bytes.Buffer{}
for {
if err := rec.Read(&suite.connMock.writeBuffer); err != nil {
break
}
suite.Equal(record.TypeApplicationData, rec.Type)
suite.Equal(record.Version12, rec.Version)
rec.Payload.WriteTo(buf) //nolint: errcheck
}
suite.Equal(dataToRec, buf.Bytes())
}
func TestConn(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnTestSuite{})
}
-59
View File
@@ -1,59 +0,0 @@
package faketls
import (
"bytes"
"errors"
)
const (
// RandomLen defines a size of the random digest in TLS Hellos.
RandomLen = 32
// ClientHelloRandomOffset is an offset in ClientHello record where
// random digest is started.
ClientHelloRandomOffset = 6
// ClientHelloSessionIDOffset is an offset in ClientHello record where
// SessionID is started.
ClientHelloSessionIDOffset = ClientHelloRandomOffset + RandomLen
// ClientHelloMinLen is a minimal possible length of
// ClientHello record.
ClientHelloMinLen = 6
// WelcomePacketRandomOffset is an offset of random in ServerHello
// packet (including record envelope).
WelcomePacketRandomOffset = 11
// HandshakeTypeClient is a value representing a client handshake.
HandshakeTypeClient = 0x01
// HandshakeTypeServer is a value representing a server handshake.
HandshakeTypeServer = 0x02
// ChangeCipherValue is a value representing a change cipher
// specification record.
ChangeCipherValue = 0x01
// ExtensionSNI is a value for TLS extension 'SNI'.
ExtensionSNI = 0x00
)
var (
// ErrBadDigest is returned if given TLS Client Hello mismatches with a
// derived one.
ErrBadDigest = errors.New("bad digest")
serverHelloSuffix = []byte{
0x00, // no compression
0x00, 0x2e, // 46 bytes of data
0x00, 0x2b, // Extension - Supported Versions
0x00, 0x02, // 2 bytes are following
0x03, 0x04, // TLS 1.3
0x00, 0x33, // Extension - Key Share
0x00, 0x24, // 36 bytes
0x00, 0x1d, // x25519 curve
0x00, 0x20, // 32 bytes of key
}
clientHelloEmptyRandom = bytes.Repeat([]byte{0}, RandomLen)
)
-84
View File
@@ -1,84 +0,0 @@
package record
import "fmt"
const TLSMaxRecordSize = 65535 // max uint16
type Type uint8
const (
// TypeChangeCipherSpec defines a byte value of the TLS record when a
// peer wants to change a specifications of the chosen cipher.
TypeChangeCipherSpec Type = 0x14
// TypeHandshake defines a byte value of the TLS record when a peer
// initiates a new TLS connection and wants to make a handshake
// ceremony.
TypeHandshake Type = 0x16
// TypeApplicationData defines a byte value of the TLS record when a
// peer sends an user data, not a control frames.
TypeApplicationData Type = 0x17
)
func (t Type) String() string {
switch t {
case TypeChangeCipherSpec:
return "changeCipher(0x14)"
case TypeHandshake:
return "handshake(0x16)"
case TypeApplicationData:
return "applicationData(0x17)"
}
return fmt.Sprintf("unknown(%#x)", byte(t))
}
func (t Type) Valid() error {
switch t {
case TypeChangeCipherSpec, TypeHandshake, TypeApplicationData:
return nil
}
return fmt.Errorf("unknown type %#x", byte(t))
}
type Version uint16
const (
// Version10 defines a TLS1.0.
Version10 Version = 769 // 0x03 0x01
// Version11 defines a TLS1.1.
Version11 Version = 770 // 0x03 0x02
// Version12 defines a TLS1.2.
Version12 Version = 771 // 0x03 0x03
// Version13 defines a TLS1.3.
Version13 Version = 772 // 0x03 0x04
)
func (v Version) String() string {
switch v {
case Version10:
return "tls1.0"
case Version11:
return "tls1.1"
case Version12:
return "tls1.2"
case Version13:
return "tls1.3"
}
return fmt.Sprintf("tls?(%d)", uint16(v))
}
func (v Version) Valid() error {
switch v {
case Version10, Version11, Version12, Version13:
return nil
}
return fmt.Errorf("unknown version %d", uint16(v))
}
@@ -1,79 +0,0 @@
package record_test
import (
"testing"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls/record"
"github.com/stretchr/testify/suite"
)
type TypeTestSuite struct {
suite.Suite
}
func (suite *TypeTestSuite) TestChangeCipherSpec() {
suite.Contains(record.TypeChangeCipherSpec.String(), "changeCipher")
suite.Contains(record.TypeChangeCipherSpec.String(), "0x14")
suite.NoError(record.TypeChangeCipherSpec.Valid())
}
func (suite *TypeTestSuite) TestHandshake() {
suite.Contains(record.TypeHandshake.String(), "handshake")
suite.Contains(record.TypeHandshake.String(), "0x16")
suite.NoError(record.TypeHandshake.Valid())
}
func (suite *TypeTestSuite) TestApplicationData() {
suite.Contains(record.TypeApplicationData.String(), "applicationData")
suite.Contains(record.TypeApplicationData.String(), "0x17")
suite.NoError(record.TypeApplicationData.Valid())
}
func (suite *TypeTestSuite) TestUnknown() {
value := record.Type(0x20)
suite.Contains(value.String(), "unknown")
suite.Contains(value.String(), "0x20")
suite.Error(value.Valid())
}
type VersionTestSuite struct {
suite.Suite
}
func (suite *VersionTestSuite) Test10() {
suite.Equal("tls1.0", record.Version10.String())
suite.NoError(record.Version10.Valid())
}
func (suite *VersionTestSuite) Test11() {
suite.Equal("tls1.1", record.Version11.String())
suite.NoError(record.Version11.Valid())
}
func (suite *VersionTestSuite) Test12() {
suite.Equal("tls1.2", record.Version12.String())
suite.NoError(record.Version12.Valid())
}
func (suite *VersionTestSuite) Test13() {
suite.Equal("tls1.3", record.Version13.String())
suite.NoError(record.Version13.Valid())
}
func (suite *VersionTestSuite) TestUnknown() {
value := record.Version(900)
suite.Equal("tls?(900)", value.String())
suite.Error(value.Valid())
}
func TestType(t *testing.T) {
t.Parallel()
suite.Run(t, &TypeTestSuite{})
}
func TestVersion(t *testing.T) {
t.Parallel()
suite.Run(t, &VersionTestSuite{})
}
-20
View File
@@ -1,20 +0,0 @@
package record
import (
"sync"
)
var recordPool = sync.Pool{
New: func() any {
return &Record{}
},
}
func AcquireRecord() *Record {
return recordPool.Get().(*Record) //nolint: forcetypeassert
}
func ReleaseRecord(r *Record) {
r.Reset()
recordPool.Put(r)
}
-86
View File
@@ -1,86 +0,0 @@
package record
import (
"bytes"
"encoding/base64"
"encoding/binary"
"fmt"
"io"
)
type Record struct {
Type Type
Version Version
Payload bytes.Buffer
}
func (r *Record) String() string {
return fmt.Sprintf("<tlsRecord(type=%v, version=%v, payload=%s)>",
r.Type,
r.Version,
base64.StdEncoding.EncodeToString(r.Payload.Bytes()))
}
func (r *Record) Reset() {
r.Payload.Reset()
}
func (r *Record) Read(reader io.Reader) error {
r.Reset()
buf := [2]byte{}
if _, err := io.ReadFull(reader, buf[:1]); err != nil {
return fmt.Errorf("cannot read type: %w", err)
}
r.Type = Type(buf[0])
if err := r.Type.Valid(); err != nil {
return fmt.Errorf("invalid type: %w", err)
}
if _, err := io.ReadFull(reader, buf[:]); err != nil {
return fmt.Errorf("cannot read version: %w", err)
}
r.Version = Version(binary.BigEndian.Uint16(buf[:]))
if err := r.Version.Valid(); err != nil {
return fmt.Errorf("invalid version: %w", err)
}
if _, err := io.ReadFull(reader, buf[:]); err != nil {
return fmt.Errorf("cannot read payload length: %w", err)
}
length := int64(binary.BigEndian.Uint16(buf[:]))
if _, err := io.CopyN(&r.Payload, reader, length); err != nil {
return fmt.Errorf("cannot read payload: %w", err)
}
return nil
}
func (r *Record) Dump(writer io.Writer) error {
buf := [2]byte{byte(r.Type), 0}
if _, err := writer.Write(buf[:1]); err != nil {
return fmt.Errorf("cannot dump record type: %w", err)
}
binary.BigEndian.PutUint16(buf[:], uint16(r.Version))
if _, err := writer.Write(buf[:]); err != nil {
return fmt.Errorf("cannot dump version: %w", err)
}
binary.BigEndian.PutUint16(buf[:], uint16(r.Payload.Len()))
if _, err := writer.Write(buf[:]); err != nil {
return fmt.Errorf("cannot dump payload length: %w", err)
}
if _, err := writer.Write(r.Payload.Bytes()); err != nil {
return fmt.Errorf("cannot dump record: %w", err)
}
return nil
}
@@ -1,110 +0,0 @@
package record_test
import (
"bytes"
"encoding/base64"
"encoding/json"
"os"
"path/filepath"
"testing"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls/record"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/suite"
)
type RecordTestSnapshot struct {
Type int `json:"type"`
Version int `json:"version"`
Payload string `json:"payload"`
Record string `json:"record"`
}
func (r RecordTestSnapshot) RecordBytes() []byte {
data, _ := base64.StdEncoding.DecodeString(r.Record)
return data
}
func (r RecordTestSnapshot) PayloadBytes() []byte {
data, _ := base64.StdEncoding.DecodeString(r.Payload)
return data
}
type RecordTestSuite struct {
suite.Suite
r *record.Record
buf *bytes.Buffer
}
func (suite *RecordTestSuite) SetupTest() {
suite.r = record.AcquireRecord()
suite.buf = &bytes.Buffer{}
}
func (suite *RecordTestSuite) TearDownTest() {
record.ReleaseRecord(suite.r)
suite.buf.Reset()
}
func (suite *RecordTestSuite) TestIdempotent() {
suite.r.Type = record.TypeApplicationData
suite.r.Version = record.Version13
suite.r.Payload.Write([]byte{1, 2, 3})
suite.NoError(suite.r.Dump(suite.buf))
suite.r.Reset()
suite.NoError(suite.r.Read(suite.buf))
suite.Equal(0, suite.buf.Len())
suite.Equal(record.TypeApplicationData, suite.r.Type)
suite.Equal(record.Version13, suite.r.Version)
suite.Equal([]byte{1, 2, 3}, suite.r.Payload.Bytes())
}
func (suite *RecordTestSuite) TestString() {
_ = suite.r.String()
}
func (suite *RecordTestSuite) TestSnapshot() {
files, err := os.ReadDir("testdata")
suite.NoError(err)
testData := map[string]string{}
for _, f := range files {
testData[f.Name()] = filepath.Join("testdata", f.Name())
}
for name, pathV := range testData {
path := pathV
suite.T().Run(name, func(t *testing.T) {
data, err := os.ReadFile(path)
assert.NoError(t, err)
snapshot := &RecordTestSnapshot{}
assert.NoError(t, json.Unmarshal(data, snapshot))
rec := record.AcquireRecord()
defer record.ReleaseRecord(rec)
assert.NoError(t, rec.Read(bytes.NewReader(snapshot.RecordBytes())))
assert.Equal(t, snapshot.Type, int(rec.Type))
assert.Equal(t, snapshot.Version, int(rec.Version))
assert.Equal(t, snapshot.PayloadBytes(), rec.Payload.Bytes())
buf := &bytes.Buffer{}
assert.NoError(t, rec.Dump(buf))
assert.Equal(t, snapshot.RecordBytes(), buf.Bytes())
})
}
}
func TestRecord(t *testing.T) {
t.Parallel()
suite.Run(t, &RecordTestSuite{})
}
@@ -1,6 +0,0 @@
{
"type": 20,
"version": 772,
"payload": "sxS+0oAyk+NBv0LLVtQOp9WSx4CweyUZPz01tQ0o4oyp8aaBl6/kMFvLq3q52KE8lCiKejLw2NxVBUkE+4izCf2gLx9qfr81opWnqJTChWzcDijvttbq9cmtDFNL+odKsS3v1/TfYEFtPsoRPrJRmOHRAnqnf49Y5Q==",
"record": "FAMEAHmzFL7SgDKT40G/QstW1A6n1ZLHgLB7JRk/PTW1DSjijKnxpoGXr+QwW8urernYoTyUKIp6MvDY3FUFSQT7iLMJ/aAvH2p+vzWilaeolMKFbNwOKO+21ur1ya0MU0v6h0qxLe/X9N9gQW0+yhE+slGY4dECeqd/j1jl"
}
@@ -1,6 +0,0 @@
{
"type": 22,
"version": 772,
"payload": "waNH223htyxCBKAb6hm0u/SK/9mhI8Ck91nfWob7QMOaIREogrDYREJH4Djcp47XrpAlEaUIDiCvoFLVJ/LK1nYs4swzfHSSl/+Aj1eqPA63XqPa8EG4FAbf0DwjwXxV9qVIhvP9b2TafKbzr4Yb5GCygzFRb/zawA==",
"record": "FgMEAHnBo0fbbeG3LEIEoBvqGbS79Ir/2aEjwKT3Wd9ahvtAw5ohESiCsNhEQkfgONynjteukCURpQgOIK+gUtUn8srWdizizDN8dJKX/4CPV6o8Drdeo9rwQbgUBt/QPCPBfFX2pUiG8/1vZNp8pvOvhhvkYLKDMVFv/NrA"
}
@@ -1,6 +0,0 @@
{
"type": 23,
"version": 769,
"payload": "jmJ0o1E5+ehAHHYAbCo4AMV03X7RSivYl250s06nD9CO44fyjaoGELz0N7IeCg1jFKcRVSCRmYYmiIY9wydn2fXOJhKif8B0BlM3qhbethYgyP+l1S8hyyETpIiOtiiiOnAJwl1D1j9OryFiJFSdRRXReIMZ4CPqPg==",
"record": "FwMBAHmOYnSjUTn56EAcdgBsKjgAxXTdftFKK9iXbnSzTqcP0I7jh/KNqgYQvPQ3sh4KDWMUpxFVIJGZhiaIhj3DJ2fZ9c4mEqJ/wHQGUzeqFt62FiDI/6XVLyHLIROkiI62KKI6cAnCXUPWP06vIWIkVJ1FFdF4gxngI+o+"
}
@@ -1,6 +0,0 @@
{
"type": 22,
"version": 769,
"payload": "hBnpBnNUdlqe/rKXa7Judcz79u7AkUgSGOycn8EqvbkZpVxnI31rNOvAsPZqG+GF7DWJ3R7H2ETmFmrpnyyng32MjSs1jptmV1oAs63zTADD7sVipgid9AJHwfl4CrC3FIQr43IPMYd29JPOl5bqu/SfrgI16PBiJw==",
"record": "FgMBAHmEGekGc1R2Wp7+spdrsm51zPv27sCRSBIY7JyfwSq9uRmlXGcjfWs068Cw9mob4YXsNYndHsfYROYWaumfLKeDfYyNKzWOm2ZXWgCzrfNMAMPuxWKmCJ30AkfB+XgKsLcUhCvjcg8xh3b0k86Xluq79J+uAjXo8GIn"
}
@@ -1,6 +0,0 @@
{
"type": 23,
"version": 770,
"payload": "Vm/C+DO56czlbtR915aHzsugSyDtp8CtojF9w1jKY0efyyfcLrNuhNg/pZm3gQ7v2BBbL1UJ97v/RIjST+5gRIfg3bBN1BE9hkf+N2AYY2lHLi0yeInHB0zFWPeHscsDopDFadIi5KtC8HvbEMuK+kK8POVk5tN9UQ==",
"record": "FwMCAHlWb8L4M7npzOVu1H3XlofOy6BLIO2nwK2iMX3DWMpjR5/LJ9wus26E2D+lmbeBDu/YEFsvVQn3u/9EiNJP7mBEh+DdsE3UET2GR/43YBhjaUcuLTJ4iccHTMVY94exywOikMVp0iLkq0Lwe9sQy4r6Qrw85WTm031R"
}
@@ -1,6 +0,0 @@
{
"type": 22,
"version": 770,
"payload": "ajPzpsgk4gwm2stRQKbllvKRLdI7vmyaj1uxEJ/kKoQnQSPumdDNKD618U2Cq6PVd0/b+9YtH67Uzx1QxtpKuby5fUXqw06WUuDAQsmjq7F26EkE5FND6rQUjUPC+e1U0dF4TQzOUSS4IAkFQPAaVehUVTRxVWa/0g==",
"record": "FgMCAHlqM/OmyCTiDCbay1FApuWW8pEt0ju+bJqPW7EQn+QqhCdBI+6Z0M0oPrXxTYKro9V3T9v71i0frtTPHVDG2kq5vLl9RerDTpZS4MBCyaOrsXboSQTkU0PqtBSNQ8L57VTR0XhNDM5RJLggCQVA8BpV6FRVNHFVZr/S"
}
@@ -1,6 +0,0 @@
{
"type": 20,
"version": 771,
"payload": "d1Hiv1NYVgEDR9mtJyv9j8mg3dWqfUpeKfOsL+jzSDfVIxeDiJZFLDT50TjNW44/yEOVEX/Y/pk+wnc7E8aCEiwGwAvB+Insw1UCJ2ejt689VWLo2u4klGVKTHuOpUvdGVTc7Lo4FAt91KQSPLYB5iqxomjEv5e3Vg==",
"record": "FAMDAHl3UeK/U1hWAQNH2a0nK/2PyaDd1ap9Sl4p86wv6PNIN9UjF4OIlkUsNPnROM1bjj/IQ5URf9j+mT7CdzsTxoISLAbAC8H4iezDVQInZ6O3rz1VYuja7iSUZUpMe46lS90ZVNzsujgUC33UpBI8tgHmKrGiaMS/l7dW"
}
@@ -1,6 +0,0 @@
{
"type": 23,
"version": 771,
"payload": "wbdU1CbrzuAJDsh6CFjGyE+AFArJj/Wmsa2wtDyW0kRuE2vUO8gg+nXkg0kkoz0WnvQEOdaswfJIaVrloD78yoyeQVfBB+VUP/63vqn60v5ccaQEn0jLdxgLjiTAxKDQDxCTMRoLnFE2ZZf28zw+HfqpIxiOZs8LhQ==",
"record": "FwMDAHnBt1TUJuvO4AkOyHoIWMbIT4AUCsmP9aaxrbC0PJbSRG4Ta9Q7yCD6deSDSSSjPRae9AQ51qzB8khpWuWgPvzKjJ5BV8EH5VQ//re+qfrS/lxxpASfSMt3GAuOJMDEoNAPEJMxGgucUTZll/bzPD4d+qkjGI5mzwuF"
}
@@ -1,6 +0,0 @@
{
"type": 23,
"version": 772,
"payload": "qqnBMb1Af3zZt4DPHpVRuIiON9ODGJUNFicFjranORh67L/HI4D6HnHyycZFUSBOw2FjMBF6UialY8snOYaRKrQmQzuUNg1Ztq7yAZ+Lgj3TBarR6OMlYhEAY0Px9Xv1UuJ0YcvQx33gdM1skJ5HBR3yZvEKNJV1LA==",
"record": "FwMEAHmqqcExvUB/fNm3gM8elVG4iI4304MYlQ0WJwWOtqc5GHrsv8cjgPoecfLJxkVRIE7DYWMwEXpSJqVjyyc5hpEqtCZDO5Q2DVm2rvIBn4uCPdMFqtHo4yViEQBjQ/H1e/VS4nRhy9DHfeB0zWyQnkcFHfJm8Qo0lXUs"
}
@@ -1,6 +0,0 @@
{
"type": 20,
"version": 769,
"payload": "NEe735TuQFp7bWpFQhASas/e1XaySvus0ovXmkfCbFq334MyFHq2eDMadziXsfu/GfBjoYggvk0LgYUeoAkBNKR0dfSovjSndaqmIUonoWl+6sZObiGZkRIMwuY2q4Eaw4/iuDu/pZhjRW/iAIH+YH7cyk/1tgdJDg==",
"record": "FAMBAHk0R7vflO5AWnttakVCEBJqz97VdrJK+6zSi9eaR8JsWrffgzIUerZ4Mxp3OJex+78Z8GOhiCC+TQuBhR6gCQE0pHR19Ki+NKd1qqYhSiehaX7qxk5uIZmREgzC5jargRrDj+K4O7+lmGNFb+IAgf5gftzKT/W2B0kO"
}
@@ -1,6 +0,0 @@
{
"type": 22,
"version": 771,
"payload": "wrXjZrPm3OSyzO0klv6/G+z2PDloR/colS/RlWwQE31Vb2xm8YkEchDDKwlc/KPLD73qMoz3MQOQLtSLc8LhVYp+l7L9jz49yTaVKtBI5UuGbo09snsKxFCgCyYUBETKabATBQtiaEu/D8dmF4Yk/2ww4sEb8DwKLQ==",
"record": "FgMDAHnCteNms+bc5LLM7SSW/r8b7PY8OWhH9yiVL9GVbBATfVVvbGbxiQRyEMMrCVz8o8sPveoyjPcxA5Au1ItzwuFVin6Xsv2PPj3JNpUq0EjlS4ZujT2yewrEUKALJhQERMppsBMFC2JoS78Px2YXhiT/bDDiwRvwPAot"
}
@@ -1,6 +0,0 @@
{
"type": 20,
"version": 770,
"payload": "OU5s8Sa11hpXWEarWzFlX55IZt3Eo+F4AMbQ/2RwB4rfHS/JNl8n63OR4oYs9QXw3RfCrYJuU9n6Xn+I/+7ZzAgZ0PbLSXW1PrLtttdfmhTErK90b49YEWdY9na4g++NMkKykwgXvY1hNxZIHX/qawEWJgxXUR3DdQ==",
"record": "FAMCAHk5TmzxJrXWGldYRqtbMWVfnkhm3cSj4XgAxtD/ZHAHit8dL8k2Xyfrc5Hihiz1BfDdF8Ktgm5T2fpef4j/7tnMCBnQ9stJdbU+su2211+aFMSsr3Rvj1gRZ1j2driD740yQrKTCBe9jWE3Fkgdf+prARYmDFdRHcN1"
}
@@ -1,8 +0,0 @@
{
"time": 1617181365,
"random": "XvCPc3aAbHbhRLv0kUmy6BfPZOGvsused5/HNsKXEPs=",
"sessionId": "St2BZ2uHMFn3B2trD1jfdtpjoJOOg6JBeLhFcyCMCq4=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "AQAB/AMDXvCPc3aAbHbhRLv0kUmy6BfPZOGvsused5/HNsKXEPsgSt2BZ2uHMFn3B2trD1jfdtpjoJOOg6JBeLhFcyCACq4ANBMDEwETAsAswCvAJMAjwArACcypwDDAL8AowCfAFMATzKgAnQCcAD0APAA1AC/ACMASAAoBAAF//wEAAQAAAAAbABkAABZzdG9yYWdlLmdvb2dsZWFwaXMuY29tABcAAAANABgAFgQDCAQEAQUDAgMIBQgFBQEIBgYBAgEABQAFANAAAAAzdAAAABIAAAAQADAALgJoMgVoMi0xNgVoMi0xNQVoMi0xNAhzcGR5LzMuMQZzcGR5LzMIaHR0cC8xLjEACwACAQAAMwAmACQAHQAgB/7oLx9JElIALsLJS91H2QNyU1H0osKwIUelVndsLyIALQACAQEAKwAJCAMEAwMDAgMBAAoACgAIAB0AFwAYABkAFQChAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
}
@@ -1,8 +0,0 @@
{
"time": 1617181365,
"random": "XvCPc3aAbHbhRLv0kUmy6BfPZOGvsused5/HNsKXEPs=",
"sessionId": "St2BZ2uHMFn3B2trD1jfdtpjoJOOg6JBeLhFcyCMCq4=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "AQAB/AMDXvCPc3aAbHbhRLv0kUmy6BfPZOGvsused5/HNsKXEPsgSt2BZ2uHMFn3B2trD1jfdtpjoJOOg6JBeLhFcyCMCq4ANBMDEwETAsAswCvAJMAjwArACcypwDDAL8AowCfAFMATzKgAnQCcAD0APAA1AC/ACMASAAoBAAF//wEAAQAAAAAbABkAABZzdG9yYWdlLmdvb2dsZWFwaXMuY29tABcAAAANABgAFgQDCAQEAQUDAgMIBQgFBQEIBgYBAgEABQAFAQAAAAAzdAAAABIAAAAQADAALgJoMgVoMi0xNgVoMi0xNQVoMi0xNAhzcGR5LzMuMQZzcGR5LzMIaHR0cC8xLjEACwACAQAAMwAmACQAHQAgB/7oLx9JElIALsLJS91H2QNyU1H0osKwIUelVndsLyIALQACAQEAKwAJCAMEAwMDAgMBAAoACgAIAB0AFwAYABkAFQChAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
}
@@ -1,8 +0,0 @@
{
"time": 1617181352,
"random": "oYEu33jl+zQbUKMtQbV1OHB0gXIM2y2aq9iY0QX12os=",
"sessionId": "FGqA3ZFYrSlj//xl7lammNn64K9/MK2mQ3HJUGvP+8g=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "AQAB/AMDoYEu33jl+zQbUKMtQbV1OHB0gXIM2y2aq9iY0QX12osgFGqA3ZFYrSlj//xl7lammNn64K9/MK2mQ3HJUGvP+8gANBMDEwETAsAswCvAJMAjwArACcypwDDAL8AowCfAFMATzKgAnQCcAD0APAA1AC/ACMASAAoBAAF//wEAAQAAAAAbABkAABZzdG9yYWdlLmdvb2dsZWFwaXMuY29tABcAAAANABgAFgQDCAQEAQUDAgMIBQgFBQEIBgYBAgEABQAFAQAAAAAzdAAAABIAAAAQADAALgJoMgVoMi0xNgVoMi0xNQVoMi0xNAhzcGR5LzMuMQZzcGR5LzMIaHR0cC8xLjEACwACAQAAMwAmACQAHQAga6CocpFP8Qd4YCFR9pkaCr97po2ALj0P5nI9Nnb3UWMALQACAQEAKwAJCAMEAwMDAgMBAAoACgAIAB0AFwAYABkAFQChAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
}
@@ -1,8 +0,0 @@
{
"time": 1617181352,
"random": "5V5sSprk/tFIgy+x1BeKNGhLlFkqfggLpgN7GYOA1ro=",
"sessionId": "jxr4d6PXPDk+Lwx3WUp9wvj8TGlOxEdrRJ0ydyJ9+H8=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "AQAB/AMD5V5sSprk/tFIgy+x1BeKNGhLlFkqfggLpgN7GYOA1rogjxr4d6PXPDk+Lwx3WUp9wvj8TGlOxEdrRJ0ydyJ9+H8ANBMDEwETAsAswCvAJMAjwArACcypwDDAL8AowCfAFMATzKgAnQCcAD0APAA1AC/ACMASAAoBAAF//wEAAQAAAAAbABkAABZzdG9yYWdlLmdvb2dsZWFwaXMuY29tABcAAAANABgAFgQDCAQEAQUDAgMIBQgFBQEIBgYBAgEABQAFAQAAAAAzdAAAABIAAAAQADAALgJoMgVoMi0xNgVoMi0xNQVoMi0xNAhzcGR5LzMuMQZzcGR5LzMIaHR0cC8xLjEACwACAQAAMwAmACQAHQAgrulAaqUdKeVYM0F+pu6on/h6LBpOyzOKG4xFIKcoFk4ALQACAQEAKwAJCAMEAwMDAgMBAAoACgAIAB0AFwAYABkAFQChAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
}
@@ -1,8 +0,0 @@
{
"time": 1617181365,
"random": "8xljlOhkDlkafEF5vu3e1r3fWvh8AX548wC3hLZ3szQ=",
"sessionId": "00uvDYKnFyZFKyf3HlLwWGCOyeHsPFiU5UZ+Fs5pDAU=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "AQAB/AMD8xljlOhkDlkafEF5vu3e1r3fWvh8AX548wC3hLZ3szQg00uvDYKnFyZFKyf3HlLwWGCOyeHsPFiU5UZ+Fs5pDAUANBMDEwETAsAswCvAJMAjwArACcypwDDAL8AowCfAFMATzKgAnQCcAD0APAA1AC/ACMASAAoBAAF//wEAAQAAAAAbABkAABZzdG9yYWdlLmdvb2dsZWFwaXMuY29tABcAAAANABgAFgQDCAQEAQUDAgMIBQgFBQEIBgYBAgEABQAFAQAAAAAzdAAAABIAAAAQADAALgJoMgVoMi0xNgVoMi0xNQVoMi0xNAhzcGR5LzMuMQZzcGR5LzMIaHR0cC8xLjEACwACAQAAMwAmACQAHQAg/9P7140NtKzjyDwBf99mOy1+FjRPAPHTNQ9WxHOKpV4ALQACAQEAKwAJCAMEAwMDAgMBAAoACgAIAB0AFwAYABkAFQChAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
}
@@ -1,8 +0,0 @@
{
"time": 1617181352,
"random": "zja3MLZ8WGSfsQRtPV75+tY6gbK3zKPi1Sy7SBBafg4=",
"sessionId": "qPut2yMqXa9zGLII/872SQ3d4Tfqo0uoDb7tpkRfBnA=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "AQAB/AMDzja3MLZ8WGSfsQRtPV75+tY6gbK3zKPi1Sy7SBBafg4gqPut2yMqXa9zGLII/872SQ3d4Tfqo0uoDb7tpkRfBnAANBMDEwETAsAswCvAJMAjwArACcypwDDAL8AowCfAFMATzKgAnQCcAD0APAA1AC/ACMASAAoBAAF//wEAAQAAAAAbABkAABZzdG9yYWdlLmdvb2dsZWFwaXMuY29tABcAAAANABgAFgQDCAQEAQUDAgMIBQgFBQEIBgYBAgEABQAFAQAAAAAzdAAAABIAAAAQADAALgJoMgVoMi0xNgVoMi0xNQVoMi0xNAhzcGR5LzMuMQZzcGR5LzMIaHR0cC8xLjEACwACAQAAMwAmACQAHQAgXviLRAqAYJ8xOLdlcsUhldI4Xl0g/s9+y2Qrd8raPEgALQACAQEAKwAJCAMEAwMDAgMBAAoACgAIAB0AFwAYABkAFQChAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
}
-91
View File
@@ -1,91 +0,0 @@
package faketls
import (
"bytes"
"crypto/hmac"
"crypto/rand"
"crypto/sha256"
"encoding/binary"
"io"
mrand "math/rand/v2"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls/record"
"golang.org/x/crypto/curve25519"
)
func SendWelcomePacket(writer io.Writer, secret []byte, clientHello ClientHello) error {
buf := &bytes.Buffer{}
rec := record.AcquireRecord()
defer record.ReleaseRecord(rec)
rec.Type = record.TypeHandshake
rec.Version = record.Version12
generateServerHello(&rec.Payload, clientHello)
rec.Dump(buf) //nolint: errcheck
rec.Reset()
rec.Type = record.TypeChangeCipherSpec
rec.Version = record.Version12
rec.Payload.WriteByte(ChangeCipherValue)
rec.Dump(buf) //nolint: errcheck
rec.Reset()
rec.Type = record.TypeApplicationData
rec.Version = record.Version12
if _, err := io.CopyN(&rec.Payload, rand.Reader, int64(1024+mrand.IntN(3092))); err != nil {
panic(err)
}
rec.Dump(buf) //nolint: errcheck
packet := buf.Bytes()
mac := hmac.New(sha256.New, secret)
mac.Write(clientHello.Random[:])
mac.Write(packet)
copy(packet[WelcomePacketRandomOffset:], mac.Sum(nil))
if _, err := writer.Write(packet); err != nil {
return err //nolint: wrapcheck
}
return nil
}
func generateServerHello(writer io.Writer, clientHello ClientHello) {
bodyBuf := &bytes.Buffer{}
sliceBuf := [2]byte{}
digest := [RandomLen]byte{}
binary.BigEndian.PutUint16(sliceBuf[:], uint16(record.Version12))
bodyBuf.Write(sliceBuf[:])
bodyBuf.Write(digest[:])
bodyBuf.WriteByte(byte(len(clientHello.SessionID)))
bodyBuf.Write(clientHello.SessionID)
binary.BigEndian.PutUint16(sliceBuf[:], clientHello.CipherSuite)
bodyBuf.Write(sliceBuf[:])
bodyBuf.Write(serverHelloSuffix)
scalar := [32]byte{}
if _, err := rand.Read(scalar[:]); err != nil {
panic(err)
}
curve, _ := curve25519.X25519(scalar[:], curve25519.Basepoint)
bodyBuf.Write(curve)
header := [4]byte{0, 0, 0, 0}
binary.BigEndian.PutUint32(header[:], uint32(bodyBuf.Len()))
header[0] = HandshakeTypeServer
writer.Write(header[:]) //nolint: errcheck
bodyBuf.WriteTo(writer) //nolint: errcheck
}
-82
View File
@@ -1,82 +0,0 @@
package faketls_test
import (
"bytes"
"crypto/hmac"
"crypto/rand"
"crypto/sha256"
"testing"
"time"
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls"
"github.com/9seconds/mtg/v2/mtglib/internal/faketls/record"
"github.com/stretchr/testify/suite"
)
type WelcomeTestSuite struct {
suite.Suite
h *faketls.ClientHello
buf *bytes.Buffer
secret mtglib.Secret
}
func (suite *WelcomeTestSuite) SetupTest() {
suite.h = &faketls.ClientHello{
Time: time.Now(),
Host: "google.com",
CipherSuite: 4867,
SessionID: make([]byte, 32),
}
_, err := rand.Read(suite.h.SessionID) //nolint: staticcheck
suite.NoError(err)
_, err = rand.Read(suite.h.Random[:]) //nolint: staticcheck
suite.NoError(err)
suite.buf = &bytes.Buffer{}
suite.secret = mtglib.GenerateSecret("google.com")
}
func (suite *WelcomeTestSuite) TestOk() {
suite.NoError(faketls.SendWelcomePacket(suite.buf, suite.secret.Key[:], *suite.h))
welcomePacket := []byte{}
welcomePacket = append(welcomePacket, suite.buf.Bytes()...)
rec := record.AcquireRecord()
defer record.ReleaseRecord(rec)
suite.NoError(rec.Read(suite.buf))
suite.Equal(record.TypeHandshake, rec.Type)
suite.Equal(record.Version12, rec.Version)
suite.NoError(rec.Read(suite.buf))
suite.Equal(record.TypeChangeCipherSpec, rec.Type)
suite.Equal(record.Version12, rec.Version)
suite.NoError(rec.Read(suite.buf))
suite.Equal(record.TypeApplicationData, rec.Type)
suite.Equal(record.Version12, rec.Version)
suite.Empty(suite.buf.Bytes())
random := make([]byte, 32)
copy(random, welcomePacket[11:])
empty := make([]byte, 32)
copy(welcomePacket[11:], empty)
mac := hmac.New(sha256.New, suite.secret.Key[:])
mac.Write(suite.h.Random[:])
mac.Write(welcomePacket)
suite.Equal(random, mac.Sum(nil))
}
func TestWelcome(t *testing.T) {
t.Parallel()
suite.Run(t, &WelcomeTestSuite{})
}
-4
View File
@@ -1,9 +1,5 @@
package relay
const (
copyBufferSize = 64 * 1024
)
type Logger interface {
Printf(msg string, args ...any)
}
@@ -0,0 +1,13 @@
//go:build mips || mipsle
package relay
import "github.com/9seconds/mtg/v2/mtglib/internal/tls"
const (
// MIPS is quite short in resources, and usually it means that it will run
// on Microtiks, OpenWRT-based routers or similar hardware. I think it worth
// to sacrifice a number of read syscalls (read, CPU load) to shrink
// limited RAM resources.
bufPoolSize = tls.MaxRecordPayloadSize / 2
)
@@ -0,0 +1,9 @@
//go:build !mips && !mipsle
package relay
import "github.com/9seconds/mtg/v2/mtglib/internal/tls"
const (
bufPoolSize = tls.MaxRecordPayloadSize
)
+18
View File
@@ -0,0 +1,18 @@
package relay
import "sync"
var bufPool = sync.Pool{
New: func() any {
b := make([]byte, bufPoolSize)
return &b
},
}
func acquireBuffer() *[]byte {
return bufPool.Get().(*[]byte)
}
func releaseBuffer(p *[]byte) {
bufPool.Put(p)
}
+6 -5
View File
@@ -15,11 +15,11 @@ func Relay(ctx context.Context, log Logger, telegramConn, clientConn essentials.
ctx, cancel := context.WithCancel(ctx)
defer cancel()
go func() {
<-ctx.Done()
stop := context.AfterFunc(ctx, func() {
telegramConn.Close() //nolint: errcheck
clientConn.Close() //nolint: errcheck
}()
})
defer stop()
closeChan := make(chan struct{})
@@ -35,12 +35,13 @@ func Relay(ctx context.Context, log Logger, telegramConn, clientConn essentials.
}
func pump(log Logger, src, dst essentials.Conn, direction string) {
var buf [copyBufferSize]byte
buf := acquireBuffer()
defer releaseBuffer(buf)
defer src.CloseRead() //nolint: errcheck
defer dst.CloseWrite() //nolint: errcheck
n, err := io.CopyBuffer(src, dst, buf[:])
n, err := io.CopyBuffer(src, dst, *buf)
switch {
case err == nil:
+84
View File
@@ -0,0 +1,84 @@
package tls
import (
"bufio"
"bytes"
"github.com/9seconds/mtg/v2/essentials"
)
const (
SizeRecordType = 1
SizeVersion = 2
SizeSize = 2
SizeHeader = SizeRecordType + SizeVersion + SizeSize
MaxRecordSize = 16384
MaxRecordPayloadSize = MaxRecordSize - SizeHeader
DefaultBufferSize = 4096
TypeChangeCipherSpec = 0x14
TypeHandshake = 0x16
TypeApplicationData = 0x17
)
// TLS 1.2 is used for both TLS 1.2 and 1.3
var TLSVersion = [SizeVersion]byte{3, 3}
// Conn presents an established TLS 1.3 connection, after handshake
type Conn struct {
essentials.Conn
p *connPayload
}
type connPayload struct {
readBuf bytes.Buffer
connBuffered *bufio.Reader
read bool
write bool
}
func (c Conn) Write(p []byte) (int, error) {
if !c.p.write {
return c.Conn.Write(p)
}
return len(p), WriteRecord(c.Conn, p)
}
func (c Conn) Read(p []byte) (int, error) {
if !c.p.read {
return c.Conn.Read(p)
}
for {
if n, err := c.p.readBuf.Read(p); err == nil {
return n, nil
}
recordType, _, err := ReadRecord(c.p.connBuffered, &c.p.readBuf)
if err != nil {
return 0, err
}
if recordType != TypeApplicationData {
c.p.readBuf.Reset()
}
}
}
func New(conn essentials.Conn, read, write bool) Conn {
newConn := Conn{
Conn: conn,
p: &connPayload{
connBuffered: bufio.NewReaderSize(conn, DefaultBufferSize),
read: read,
write: write,
},
}
newConn.p.readBuf.Grow(DefaultBufferSize)
return newConn
}
+160
View File
@@ -0,0 +1,160 @@
package tls
import (
"io"
"testing"
"github.com/9seconds/mtg/v2/internal/testlib"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/suite"
)
type ConnTestSuite struct {
suite.Suite
connMock *testlib.EssentialsConnMock
}
func (suite *ConnTestSuite) SetupTest() {
suite.connMock = &testlib.EssentialsConnMock{}
}
func (suite *ConnTestSuite) TearDownTest() {
suite.connMock.AssertExpectations(suite.T())
}
func (suite *ConnTestSuite) feedRead(raw []byte) {
suite.connMock.
On("Read", mock.AnythingOfType("[]uint8")).
Run(func(args mock.Arguments) {
copy(args.Get(0).([]byte), raw)
}).
Return(len(raw), nil).
Once()
suite.connMock.
On("Read", mock.AnythingOfType("[]uint8")).
Return(0, io.EOF).
Maybe()
}
func (suite *ConnTestSuite) TestReadTLSEnabled() {
payload := []byte("hello world")
suite.feedRead(MakeTLSRecord(0x17, payload))
conn := New(suite.connMock, true, false)
buf := make([]byte, 128)
n, err := conn.Read(buf)
suite.NoError(err)
suite.Equal(payload, buf[:n])
}
func (suite *ConnTestSuite) TestReadTLSSkipsNonApplicationData() {
raw := append(
MakeTLSRecord(0x14, []byte{1}),
MakeTLSRecord(0x17, []byte("real data"))...,
)
suite.feedRead(raw)
conn := New(suite.connMock, true, false)
buf := make([]byte, 128)
n, err := conn.Read(buf)
suite.NoError(err)
suite.Equal([]byte("real data"), buf[:n])
}
func (suite *ConnTestSuite) TestReadTLSMultipleRecords() {
raw := append(
MakeTLSRecord(0x17, []byte("first")),
MakeTLSRecord(0x17, []byte("second"))...,
)
suite.feedRead(raw)
conn := New(suite.connMock, true, false)
buf := make([]byte, 128)
n, err := conn.Read(buf)
suite.NoError(err)
suite.Equal([]byte("first"), buf[:n])
n, err = conn.Read(buf)
suite.NoError(err)
suite.Equal([]byte("second"), buf[:n])
}
func (suite *ConnTestSuite) TestReadTLSSmallBuffer() {
payload := []byte("hello world, this is a longer payload")
suite.feedRead(MakeTLSRecord(0x17, payload))
conn := New(suite.connMock, true, false)
small := make([]byte, 5)
n, err := conn.Read(small)
suite.NoError(err)
suite.Equal(payload[:5], small[:n])
rest := make([]byte, 128)
n, err = conn.Read(rest)
suite.NoError(err)
suite.Equal(payload[5:], rest[:n])
}
func (suite *ConnTestSuite) TestReadPassthrough() {
data := []byte("raw bytes")
suite.connMock.
On("Read", mock.AnythingOfType("[]uint8")).
Run(func(args mock.Arguments) {
copy(args.Get(0).([]byte), data)
}).
Return(len(data), nil).
Once()
conn := New(suite.connMock, false, false)
buf := make([]byte, 128)
n, err := conn.Read(buf)
suite.NoError(err)
suite.Equal(data, buf[:n])
}
func (suite *ConnTestSuite) TestWritePassthrough() {
data := []byte("outgoing data")
suite.connMock.
On("Write", mock.AnythingOfType("[]uint8")).
Return(len(data), nil).
Once()
conn := New(suite.connMock, false, false)
n, err := conn.Write(data)
suite.NoError(err)
suite.Equal(len(data), n)
}
func (suite *ConnTestSuite) TestWriteTLSEnabled() {
data := []byte("outgoing data")
suite.connMock.
On("Write", mock.AnythingOfType("[]uint8")).
Return(len(data), nil).
Once()
conn := New(suite.connMock, false, true)
n, err := conn.Write(data)
suite.NoError(err)
suite.Equal(len(data), n)
}
func TestConn(t *testing.T) {
t.Parallel()
suite.Run(t, &ConnTestSuite{})
}
+21
View File
@@ -0,0 +1,21 @@
package fake
import (
"bytes"
"sync"
)
var bytesPool = sync.Pool{
New: func() any {
return &bytes.Buffer{}
},
}
func acquireBuffer() *bytes.Buffer {
return bytesPool.Get().(*bytes.Buffer)
}
func releaseBuffer(b *bytes.Buffer) {
b.Reset()
bytesPool.Put(b)
}
+261
View File
@@ -0,0 +1,261 @@
package fake
import (
"bytes"
"crypto/hmac"
"crypto/sha256"
"crypto/subtle"
"encoding/binary"
"errors"
"fmt"
"io"
"net"
"slices"
"time"
)
const (
TypeHandshakeClient = 0x01
RandomLen = 32
// record_type(1) + version(2) + size(2) + handshake_type(1) + uint24_length(3) + client_version(2)
RandomOffset = 1 + 2 + 2 + 1 + 3 + 2
// https://datatracker.ietf.org/doc/html/rfc8701#name-grease-values
// https://medium.com/asecuritysite-when-bob-met-alice/in-cybersecurity-what-is-grease-9f8850558dea
GreaseMask = 0x0f0f
GreaseValueType = 0x0a0a
sniDNSNamesListType = 0
)
var (
emptyRandom = [RandomLen]byte{}
extTypeSNI = [2]byte{}
ErrCannotFindCipher = errors.New("cannot find a cipher")
)
type ClientHello struct {
Random [RandomLen]byte
SessionID []byte
CipherSuite uint16
}
func ReadClientHello(
conn net.Conn,
secret []byte,
hostname string,
tolerateTimeSkewness time.Duration,
) (*ClientHello, error) {
// This is how FakeTLS is organized:
// 1. We create sha256 HMAC with a given secret
// 2. We dump there a whole TLS frame except of the fact that random
// is filled with all zeroes
// 3. Digest is computed. This digest should be XORed with
// original client random
// 4. New digest should be all 0 except of last 4 bytes
// 5. Last 4 bytes are little endian uint32 of UNIX timestamp when
// this message was created.
clientHelloCopy, handshakeReader, err := parseClientHello(conn)
if err != nil {
return nil, fmt.Errorf("cannot read client hello: %w", err)
}
hello, err := parseHandshake(handshakeReader)
if err != nil {
return nil, fmt.Errorf("cannot parse handshake: %w", err)
}
sniHostnames, err := parseSNI(handshakeReader)
if err != nil {
return nil, fmt.Errorf("cannot parse SNI: %w", err)
}
if !slices.Contains(sniHostnames, hostname) {
return nil, fmt.Errorf("cannot find %s in %v", hostname, sniHostnames)
}
digest := hmac.New(sha256.New, secret)
// we write a copy of the handshake with client random all nullified.
digest.Write(clientHelloCopy.Next(RandomOffset))
clientHelloCopy.Next(RandomLen)
digest.Write(emptyRandom[:])
digest.Write(clientHelloCopy.Bytes())
computed := digest.Sum(nil)
for i := range RandomLen {
computed[i] ^= hello.Random[i]
}
if subtle.ConstantTimeCompare(emptyRandom[:RandomLen-4], computed[:RandomLen-4]) != 1 {
return nil, ErrBadDigest
}
timestamp := int64(binary.LittleEndian.Uint32(computed[RandomLen-4:]))
createdAt := time.Unix(timestamp, 0)
if tdiff := time.Since(createdAt).Abs(); tdiff > tolerateTimeSkewness {
return nil, fmt.Errorf("timestamp %q is too old %s", createdAt, tdiff)
}
return hello, nil
}
func parseHandshake(r io.Reader) (*ClientHello, error) {
// A protocol version of "3,3" (meaning TLS 1.2) is given.
header := [2]byte{}
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read client version: %w", err)
}
hello := &ClientHello{}
if _, err := io.ReadFull(r, hello.Random[:]); err != nil {
return nil, fmt.Errorf("cannot read client random: %w", err)
}
if _, err := io.ReadFull(r, header[:1]); err != nil {
return nil, fmt.Errorf("cannot read session ID length: %w", err)
}
hello.SessionID = make([]byte, int(header[0]))
if _, err := io.ReadFull(r, hello.SessionID); err != nil {
return nil, fmt.Errorf("cannot read session id: %w", err)
}
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read cipher suite length: %w", err)
}
cipherSuiteLen := int64(binary.BigEndian.Uint16(header[:]))
// Pick the first non-GREASE cipher suite from the list.
// Real TLS servers never select GREASE values (RFC 8701, pattern 0x?a?a),
// so echoing them back is a trivial DPI fingerprint.
// cipherSuiteLen is in bytes; each cipher suite is 2 bytes.
for range cipherSuiteLen / 2 {
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read cipher suite: %w", err)
}
if hello.CipherSuite != 0 {
// do not forget we have to scan until the end
continue
}
if cs := binary.BigEndian.Uint16(header[:]); cs&GreaseMask != GreaseValueType {
hello.CipherSuite = cs
}
}
if hello.CipherSuite == 0 {
return nil, ErrCannotFindCipher
}
if _, err := io.ReadFull(r, header[:1]); err != nil {
return nil, fmt.Errorf("cannot read compression methods length: %w", err)
}
if _, err := io.CopyN(io.Discard, r, int64(header[0])); err != nil {
return nil, fmt.Errorf("cannot skip compression methods: %w", err)
}
return hello, nil
}
func parseSNI(r io.Reader) ([]string, error) {
header := [2]byte{}
if _, err := io.ReadFull(r, header[:]); err != nil {
return nil, fmt.Errorf("cannot read length of TLS extensions: %w", err)
}
extensionsLength := int64(binary.BigEndian.Uint16(header[:]))
buf := &bytes.Buffer{}
buf.Grow(int(extensionsLength))
if _, err := io.CopyN(buf, r, extensionsLength); err != nil {
return nil, fmt.Errorf("cannot read extensions: %w", err)
}
for buf.Len() > 0 {
// 00 00 - assigned value for extension "server name"
// 00 18 - 0x18 (24) bytes of "server name" extension data follows
// 00 16 - 0x16 (22) bytes of first (and only) list entry follows
// 00 - list entry is type 0x00 "DNS hostname"
// 00 13 - 0x13 (19) bytes of hostname follows
// 65 78 61 ... 6e 65 74 - "example.ulfheim.net"
// 00 00 - assigned value for extension "server name"
extTypeB := buf.Next(2)
if len(extTypeB) != 2 {
return nil, fmt.Errorf("cannot read extension type: %v", extTypeB)
}
// 00 18 - 0x18 (24) bytes of "server name" extension data follows
lengthB := buf.Next(2)
if len(lengthB) != 2 {
return nil, fmt.Errorf("cannot read extension %v length: %v", extTypeB, lengthB)
}
length := int(binary.BigEndian.Uint16(lengthB))
extDataB := buf.Next(length)
if len(extDataB) != length {
return nil, fmt.Errorf("cannot read extension %v data: len %d != %d", extTypeB, length, len(extDataB))
}
if !bytes.Equal(extTypeB, extTypeSNI[:]) {
continue
}
buf.Reset()
buf.Write(extDataB)
// 00 16 - 0x16 (22) bytes of first (and only) list entry follows
lengthB = buf.Next(2)
if len(lengthB) != 2 {
return nil, fmt.Errorf("cannot read the length of the SNI record: %v", lengthB)
}
length = int(binary.BigEndian.Uint16(lengthB))
if length == 0 {
return nil, nil
}
listType, err := buf.ReadByte()
if err != nil {
return nil, fmt.Errorf("cannot read SNI list type: %w", err)
}
// 00 - list entry is type 0x00 "DNS hostname"
if listType != sniDNSNamesListType {
return nil, fmt.Errorf("incorrect SNI list type %#x", listType)
}
names := []string{}
for buf.Len() > 0 {
// 00 13 - 0x13 (19) bytes of hostname follows
lengthB = buf.Next(2)
if len(lengthB) != 2 {
return nil, fmt.Errorf("incorrect length of the hostname: %v", lengthB)
}
length = int(binary.BigEndian.Uint16(lengthB))
name := buf.Next(length)
if len(name) != length {
return nil, fmt.Errorf("incorrect length of SNI hostname: len %d != %d", length, len(name))
}
names = append(names, string(name))
}
return names, nil
}
return nil, nil
}
@@ -0,0 +1,48 @@
package fake_test
import (
"bytes"
"testing"
"time"
"github.com/9seconds/mtg/v2/internal/testlib"
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/tls/fake"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
)
type connMock struct {
testlib.EssentialsConnMock
readBuf *bytes.Buffer
}
func (f *connMock) Read(p []byte) (int, error) {
return f.readBuf.Read(p)
}
func FuzzReadClientHello(f *testing.F) {
seed := [248]byte{}
secret, err := mtglib.ParseSecret(
"ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d",
)
require.NoError(f, err)
f.Add(seed[:])
f.Fuzz(func(t *testing.T, value []byte) {
r := &connMock{
readBuf: bytes.NewBuffer(value),
}
r.
On("SetReadDeadline", mock.AnythingOfType("time.Time")).
Twice().
Return(nil)
_, err := fake.ReadClientHello(r, secret.Key[:], secret.Host, time.Hour)
assert.Error(t, err)
})
}
@@ -0,0 +1,147 @@
package fake_test
import (
"bytes"
"encoding/base64"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/tls/fake"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
)
type clientHelloSnapshot struct {
Time int `json:"time"`
Random string `json:"random"`
SessionID string `json:"sessionId"`
Host string `json:"host"`
CipherSuite int `json:"cipherSuite"`
Full string `json:"full"`
}
func (c clientHelloSnapshot) GetRandom() []byte {
data, _ := base64.StdEncoding.DecodeString(c.Random)
return data
}
func (c clientHelloSnapshot) GetSessionID() []byte {
data, _ := base64.StdEncoding.DecodeString(c.SessionID)
return data
}
func (c clientHelloSnapshot) GetCipherSuite() uint16 {
return uint16(c.CipherSuite)
}
func (c clientHelloSnapshot) GetFull() []byte {
data, _ := base64.StdEncoding.DecodeString(c.Full)
return data
}
type ParseClientHelloSnapshotTestSuite struct {
suite.Suite
secret mtglib.Secret
}
func (suite *ParseClientHelloSnapshotTestSuite) SetupSuite() {
parsed, err := mtglib.ParseSecret(
"ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d",
)
require.NoError(suite.T(), err)
suite.secret = parsed
}
func (suite *ParseClientHelloSnapshotTestSuite) makeConn(data []byte) *parseClientHelloConnMock {
readBuf := &bytes.Buffer{}
readBuf.Write(data)
connMock := &parseClientHelloConnMock{
readBuf: readBuf,
}
return connMock
}
func (suite *ParseClientHelloSnapshotTestSuite) TestSnapshotOk() {
files, err := os.ReadDir("testdata")
require.NoError(suite.T(), err)
for _, v := range files {
if !strings.HasPrefix(v.Name(), "client-hello-ok") {
continue
}
path := filepath.Join("testdata", v.Name())
suite.T().Run(v.Name(), func(t *testing.T) {
fileData, err := os.ReadFile(path)
assert.NoError(t, err)
snapshot := &clientHelloSnapshot{}
assert.NoError(t, json.Unmarshal(fileData, snapshot))
connMock := suite.makeConn(snapshot.GetFull())
defer connMock.AssertExpectations(t)
hello, err := fake.ReadClientHello(
connMock,
suite.secret.Key[:],
suite.secret.Host,
TolerateTime,
)
require.NoError(t, err)
assert.Equal(t, snapshot.GetRandom(), hello.Random[:])
assert.Equal(t, snapshot.GetSessionID(), hello.SessionID)
assert.Equal(t, snapshot.GetCipherSuite(), hello.CipherSuite)
})
}
}
func (suite *ParseClientHelloSnapshotTestSuite) TestSnapshotBad() {
files, err := os.ReadDir("testdata")
require.NoError(suite.T(), err)
for _, v := range files {
if !strings.HasPrefix(v.Name(), "client-hello-bad") {
continue
}
path := filepath.Join("testdata", v.Name())
suite.T().Run(v.Name(), func(t *testing.T) {
fileData, err := os.ReadFile(path)
assert.NoError(t, err)
snapshot := &clientHelloSnapshot{}
assert.NoError(t, json.Unmarshal(fileData, snapshot))
connMock := suite.makeConn(snapshot.GetFull())
defer connMock.AssertExpectations(t)
_, err = fake.ReadClientHello(
connMock,
suite.secret.Key[:],
suite.secret.Host,
TolerateTime,
)
assert.ErrorIs(t, err, fake.ErrBadDigest)
})
}
}
func TestParseClientHelloSnapshot(t *testing.T) {
t.Parallel()
suite.Run(t, &ParseClientHelloSnapshotTestSuite{})
}
@@ -0,0 +1,662 @@
package fake_test
import (
"bytes"
cryptotls "crypto/tls"
"encoding/binary"
"encoding/json"
"io"
"os"
"testing"
"time"
"github.com/9seconds/mtg/v2/internal/testlib"
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
"github.com/9seconds/mtg/v2/mtglib/internal/tls/fake"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
)
const (
TolerateTime = 365 * 30 * 24 * time.Hour
)
type parseClientHelloConnMock struct {
testlib.EssentialsConnMock
readBuf *bytes.Buffer
}
func (m *parseClientHelloConnMock) Read(p []byte) (int, error) {
return m.readBuf.Read(p)
}
type ParseClientHelloTestSuite struct {
suite.Suite
secret mtglib.Secret
readBuf *bytes.Buffer
connMock *parseClientHelloConnMock
}
func (suite *ParseClientHelloTestSuite) SetupSuite() {
parsed, err := mtglib.ParseSecret("ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d")
require.NoError(suite.T(), err)
suite.secret = parsed
}
func (suite *ParseClientHelloTestSuite) SetupTest() {
suite.readBuf = &bytes.Buffer{}
suite.connMock = &parseClientHelloConnMock{
readBuf: suite.readBuf,
}
}
func (suite *ParseClientHelloTestSuite) TearDownTest() {
suite.connMock.AssertExpectations(suite.T())
}
type ParseClientHello_TLSHeaderTestSuite struct {
ParseClientHelloTestSuite
}
func (suite *ParseClientHello_TLSHeaderTestSuite) TestEmpty() {
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read client hello")
}
func (suite *ParseClientHello_TLSHeaderTestSuite) TestNothing() {
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorIs(err, io.EOF)
}
func (suite *ParseClientHello_TLSHeaderTestSuite) TestUnknownRecord() {
suite.readBuf.Write([]byte{
10,
3, 3,
0, 0,
})
suite.readBuf.WriteByte(10)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "unexpected record type 0xa")
}
func (suite *ParseClientHello_TLSHeaderTestSuite) TestUnknownProtocolVersion() {
suite.readBuf.Write([]byte{
tls.TypeHandshake,
3, 3,
0, 0,
})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "unexpected protocol version")
}
func (suite *ParseClientHello_TLSHeaderTestSuite) TestCannotReadRestOfRecord() {
suite.readBuf.Write([]byte{
tls.TypeHandshake,
3, 1,
0, 10,
})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorIs(err, io.EOF)
}
type ParseClientHelloHandshakeTestSuite struct {
ParseClientHelloTestSuite
}
func (suite *ParseClientHelloHandshakeTestSuite) SetupTest() {
suite.ParseClientHelloTestSuite.SetupTest()
suite.readBuf.Write([]byte{
tls.TypeHandshake,
3, 1,
0,
})
}
func (suite *ParseClientHelloHandshakeTestSuite) TestCannotReadHeader() {
suite.readBuf.Write([]byte{
1,
10,
})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read handshake header")
}
func (suite *ParseClientHelloHandshakeTestSuite) TestIncorrectHandshakeType() {
suite.readBuf.Write([]byte{
4,
10, 0, 0, 0,
})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "incorrect handshake type")
}
func (suite *ParseClientHelloHandshakeTestSuite) TestCannotReadHandshake() {
suite.readBuf.Write([]byte{
4 + 3,
10, 0, 0, 0,
})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorIs(err, io.EOF)
}
type ParseClientHelloHandshakeBodyTestSuite struct {
ParseClientHelloTestSuite
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) SetupTest() {
suite.ParseClientHelloTestSuite.SetupTest()
suite.readBuf.Write([]byte{
tls.TypeHandshake,
3, 1,
0,
})
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) writeBody(body []byte) {
suite.readBuf.WriteByte(byte(4 + len(body)))
suite.readBuf.Write([]byte{
fake.TypeHandshakeClient,
0, 0, byte(len(body)),
})
suite.readBuf.Write(body)
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadVersion() {
suite.writeBody(nil)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read client version")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadRandom() {
suite.writeBody([]byte{3, 3})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read client random")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadSessionIDLength() {
body := make([]byte, 2+fake.RandomLen)
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read session ID length")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadSessionID() {
body := make([]byte, 2+fake.RandomLen+1)
body[2+fake.RandomLen] = 32
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read session id")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadCipherSuiteLength() {
body := make([]byte, 2+fake.RandomLen+1)
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read cipher suite length")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadFirstCipherSuite() {
body := make([]byte, 2+fake.RandomLen+1+2+1) // cipherSuiteLen=2 but only 1 byte available
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2)
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read cipher suite")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipRemainingCipherSuites() {
body := make([]byte, 2+fake.RandomLen+1+2+2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 4)
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read cipher suite")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotFindCipher() {
// All cipher suites are GREASE values — must return ErrCannotFindCipher.
body := make([]byte, 2+fake.RandomLen+1+2+4+1)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 4)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2:], 0x0a0a)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2+2:], 0x1a1a)
body[2+fake.RandomLen+1+2+4] = 1
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorIs(err, fake.ErrCannotFindCipher)
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotReadCompressionMethodsLength() {
body := make([]byte, 2+fake.RandomLen+1+2+2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2:], cryptotls.TLS_AES_128_GCM_SHA256)
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read compression methods length")
}
func (suite *ParseClientHelloHandshakeBodyTestSuite) TestCannotSkipCompressionMethods() {
body := make([]byte, 2+fake.RandomLen+1+2+2+1)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1:], 2)
binary.BigEndian.PutUint16(body[2+fake.RandomLen+1+2:], cryptotls.TLS_AES_128_GCM_SHA256)
body[2+fake.RandomLen+1+2+2] = 1
suite.writeBody(body)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot skip compression methods")
}
type ParseClientHelloSNITestSuite struct {
ParseClientHelloTestSuite
}
func (suite *ParseClientHelloSNITestSuite) SetupTest() {
suite.ParseClientHelloTestSuite.SetupTest()
suite.readBuf.Write([]byte{
tls.TypeHandshake,
3, 1,
0,
})
}
func (suite *ParseClientHelloSNITestSuite) writeExtensions(extensions []byte) {
handshakeBodyLen := 41 + len(extensions)
suite.readBuf.WriteByte(byte(4 + handshakeBodyLen))
suite.readBuf.Write([]byte{
fake.TypeHandshakeClient,
0, 0, byte(handshakeBodyLen),
})
// version(2) + random(32) + sessionIDLen(1) + cipherSuiteLen(2) +
// cipherSuite(2) + compressionLen(1) + compression(1) = 41
body := make([]byte, 41)
binary.BigEndian.PutUint16(body[35:], 2)
binary.BigEndian.PutUint16(body[37:], cryptotls.TLS_AES_128_GCM_SHA256)
body[39] = 1
suite.readBuf.Write(body)
suite.readBuf.Write(extensions)
}
func (suite *ParseClientHelloSNITestSuite) TestCannotReadExtensionsLength() {
suite.writeExtensions(nil)
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read length of TLS extensions")
}
func (suite *ParseClientHelloSNITestSuite) TestCannotReadExtensions() {
suite.writeExtensions([]byte{0, 10})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read extensions")
}
func (suite *ParseClientHelloSNITestSuite) TestCannotReadExtensionType() {
suite.writeExtensions([]byte{0, 1, 0xAB})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read extension type")
}
func (suite *ParseClientHelloSNITestSuite) TestCannotReadExtensionLength() {
suite.writeExtensions([]byte{0, 2, 0xFF, 0xFF})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "length:")
}
func (suite *ParseClientHelloSNITestSuite) TestCannotReadExtensionData() {
suite.writeExtensions([]byte{0, 4, 0xFF, 0xFF, 0, 5})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "data: len")
}
func (suite *ParseClientHelloSNITestSuite) TestCannotReadSNIRecordLength() {
suite.writeExtensions([]byte{0, 5, 0, 0, 0, 1, 0xAB})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read the length of the SNI record")
}
func (suite *ParseClientHelloSNITestSuite) TestCannotReadSNIListType() {
suite.writeExtensions([]byte{0, 6, 0, 0, 0, 2, 0, 1})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "cannot read SNI list type")
}
func (suite *ParseClientHelloSNITestSuite) TestIncorrectSNIListType() {
suite.writeExtensions([]byte{0, 7, 0, 0, 0, 3, 0, 1, 5})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "incorrect SNI list type")
}
func (suite *ParseClientHelloSNITestSuite) TestCannotReadHostnameLength() {
suite.writeExtensions([]byte{0, 8, 0, 0, 0, 4, 0, 2, 0, 0xAB})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "incorrect length of the hostname")
}
func (suite *ParseClientHelloSNITestSuite) TestCannotReadHostname() {
suite.writeExtensions([]byte{0, 9, 0, 0, 0, 5, 0, 3, 0, 0, 5})
_, err := fake.ReadClientHello(suite.connMock, suite.secret.Key[:], suite.secret.Host, TolerateTime)
suite.ErrorContains(err, "incorrect length of SNI hostname")
}
func TestParseClientHelloTLSHeader(t *testing.T) {
t.Parallel()
suite.Run(t, &ParseClientHello_TLSHeaderTestSuite{})
}
func TestParseClientHelloHandshake(t *testing.T) {
t.Parallel()
suite.Run(t, &ParseClientHelloHandshakeTestSuite{})
}
func TestParseClientHelloHandshakeBody(t *testing.T) {
t.Parallel()
suite.Run(t, &ParseClientHelloHandshakeBodyTestSuite{})
}
func TestParseClientHelloSNI(t *testing.T) {
t.Parallel()
suite.Run(t, &ParseClientHelloSNITestSuite{})
}
// fragmentTLSRecord splits a single TLS record into n TLS records by
// dividing the payload into roughly equal parts. Each part gets its own
// TLS record header with the same record type and version.
func fragmentTLSRecord(t testing.TB, full []byte, n int) []byte {
t.Helper()
recordType := full[0]
version := full[1:3]
payload := full[tls.SizeHeader:]
chunkSize := len(payload) / n
result := &bytes.Buffer{}
for i := 0; i < n; i++ {
start := i * chunkSize
end := start + chunkSize
if i == n-1 {
end = len(payload)
}
chunk := payload[start:end]
result.WriteByte(recordType)
result.Write(version)
require.NoError(t, binary.Write(result, binary.BigEndian, uint16(len(chunk))))
result.Write(chunk)
}
return result.Bytes()
}
// splitPayloadAt creates two TLS records from a single record by splitting
// the payload at the given byte position.
func splitPayloadAt(t testing.TB, full []byte, pos int) []byte {
t.Helper()
payload := full[tls.SizeHeader:]
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(t, binary.Write(buf, binary.BigEndian, uint16(pos)))
buf.Write(payload[:pos])
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(t, binary.Write(buf, binary.BigEndian, uint16(len(payload)-pos)))
buf.Write(payload[pos:])
return buf.Bytes()
}
type ParseClientHelloFragmentedTestSuite struct {
suite.Suite
secret mtglib.Secret
snapshot *clientHelloSnapshot
}
func (s *ParseClientHelloFragmentedTestSuite) SetupSuite() {
parsed, err := mtglib.ParseSecret(
"ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d",
)
require.NoError(s.T(), err)
s.secret = parsed
fileData, err := os.ReadFile("testdata/client-hello-ok-19dfe38384b9884b.json")
require.NoError(s.T(), err)
s.snapshot = &clientHelloSnapshot{}
require.NoError(s.T(), json.Unmarshal(fileData, s.snapshot))
}
func (s *ParseClientHelloFragmentedTestSuite) makeConn(data []byte) *parseClientHelloConnMock {
readBuf := &bytes.Buffer{}
readBuf.Write(data)
connMock := &parseClientHelloConnMock{
readBuf: readBuf,
}
return connMock
}
func (s *ParseClientHelloFragmentedTestSuite) TestReassemblySuccess() {
full := s.snapshot.GetFull()
tests := []struct {
name string
data []byte
}{
{"two equal fragments", fragmentTLSRecord(s.T(), full, 2)},
{"three equal fragments", fragmentTLSRecord(s.T(), full, 3)},
{"single byte first fragment", splitPayloadAt(s.T(), full, 1)},
{"three byte first fragment", splitPayloadAt(s.T(), full, 3)},
}
for _, tt := range tests {
s.Run(tt.name, func() {
connMock := s.makeConn(tt.data)
defer connMock.AssertExpectations(s.T())
hello, err := fake.ReadClientHello(
connMock,
s.secret.Key[:],
s.secret.Host,
TolerateTime,
)
s.Require().NoError(err)
s.Equal(s.snapshot.GetRandom(), hello.Random[:])
s.Equal(s.snapshot.GetSessionID(), hello.SessionID)
s.Equal(uint16(s.snapshot.CipherSuite), hello.CipherSuite)
})
}
}
func (s *ParseClientHelloFragmentedTestSuite) TestReassemblyErrors() {
full := s.snapshot.GetFull()
payload := full[tls.SizeHeader:]
tests := []struct {
name string
buildData func() []byte
errMsg string
}{
{
name: "wrong continuation record type",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Wrong type: application data instead of handshake
buf.WriteByte(tls.TypeApplicationData)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(payload)-10)))
buf.Write(payload[10:])
return buf.Bytes()
},
errMsg: "unexpected record type",
},
{
name: "too many continuation records",
buildData: func() []byte {
// Handshake header claiming 256 bytes, but we only send 1 byte per continuation
handshakePayload := []byte{0x01, 0x00, 0x01, 0x00}
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 1})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(handshakePayload))))
buf.Write(handshakePayload)
for range 11 {
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 1})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(1)))
buf.WriteByte(0xAB)
}
return buf.Bytes()
},
errMsg: "too many fragments",
},
{
name: "zero-length continuation record",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Valid header but zero-length payload
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(0)))
return buf.Bytes()
},
errMsg: "cannot read record header",
},
{
name: "wrong continuation record version",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Wrong version: 3.3 instead of 3.1
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 3})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(payload)-10)))
buf.Write(payload[10:])
return buf.Bytes()
},
errMsg: "unexpected protocol version",
},
{
name: "handshake message too large",
buildData: func() []byte {
// Handshake header claiming 0x010000 (65536) bytes — exceeds 0xFFFF limit
handshakePayload := []byte{0x01, 0x01, 0x00, 0x00}
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write([]byte{3, 1})
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(len(handshakePayload))))
buf.Write(handshakePayload)
return buf.Bytes()
},
errMsg: "cannot read record header",
},
{
name: "truncated continuation record header",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Connection ends mid-header (only 2 bytes)
buf.WriteByte(tls.TypeHandshake)
buf.WriteByte(3)
return buf.Bytes()
},
errMsg: "cannot read record header",
},
{
name: "truncated continuation record payload",
buildData: func() []byte {
buf := &bytes.Buffer{}
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(10)))
buf.Write(payload[:10])
// Claims 100 bytes but no payload follows
buf.WriteByte(tls.TypeHandshake)
buf.Write(full[1:3])
require.NoError(s.T(), binary.Write(buf, binary.BigEndian, uint16(100)))
return buf.Bytes()
},
errMsg: "EOF",
},
}
for _, tt := range tests {
s.Run(tt.name, func() {
connMock := s.makeConn(tt.buildData())
defer connMock.AssertExpectations(s.T())
_, err := fake.ReadClientHello(
connMock,
s.secret.Key[:],
s.secret.Host,
TolerateTime,
)
s.ErrorContains(err, tt.errMsg)
})
}
}
func TestParseClientHelloFragmented(t *testing.T) {
t.Parallel()
suite.Run(t, &ParseClientHelloFragmentedTestSuite{})
}
+7
View File
@@ -0,0 +1,7 @@
package fake
import (
"errors"
)
var ErrBadDigest = errors.New("incorrect client random")
+162
View File
@@ -0,0 +1,162 @@
package fake
import (
"bytes"
"crypto/hmac"
"crypto/rand"
"crypto/sha256"
"encoding/binary"
"io"
rnd "math/rand/v2"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
"golang.org/x/crypto/curve25519"
)
// NoiseParams controls the size of the fake ApplicationData record
// in ServerHello. If Mean is 0, the legacy random range (2500-4700)
// is used.
type NoiseParams struct {
Mean int
Jitter int
}
const (
TypeHandshakeServer = 0x02
ChangeCipherValue = 0x01
EllipticCurveLen = 32
)
var serverHelloSuffix = []byte{
0x00, // no compression
0x00, 0x2e, // 46 bytes of data
0x00, 0x2b, // Extension - Supported Versions
0x00, 0x02, // 2 bytes are following
0x03, 0x04, // TLS 1.3
0x00, 0x33, // Extension - Key Share
0x00, 0x24, // 36 bytes
0x00, 0x1d, // x25519 curve
0x00, 0x20, // 32 bytes of key
}
func SendServerHello(w io.Writer, secret []byte, clientHello *ClientHello, noise NoiseParams) error {
buf := &bytes.Buffer{}
buf.Grow(tls.MaxRecordSize)
generateServerHello(buf, clientHello)
generateChangeCipherValue(buf)
generateNoise(buf, noise)
packet := buf.Bytes()
digest := hmac.New(sha256.New, secret)
digest.Write(clientHello.Random[:])
digest.Write(packet)
copy(packet[RandomOffset:], digest.Sum(nil))
_, err := w.Write(packet)
return err
}
func generateServerHello(buf *bytes.Buffer, hello *ClientHello) {
payload := acquireBuffer()
defer releaseBuffer(payload)
generateServerHelloPayload(payload, hello)
// 16 - type is 0x16 (handshake record)
// 03 03 - legacy protocol version of "3,3" (TLS 1.2)
// 00 7a - 0x7A (122) bytes of handshake message follows
// 16 - type is 0x16 (handshake record)
buf.WriteByte(tls.TypeHandshake)
// 03 03 - legacy protocol version of "3,3" (TLS 1.2)
buf.Write(tls.TLSVersion[:])
// 00 7a - 0x7A (122) bytes of handshake message follows
binary.Write(buf, binary.BigEndian, uint16(payload.Len())) //nolint: errcheck
payload.WriteTo(buf) //nolint: errcheck
}
func generateServerHelloPayload(buf *bytes.Buffer, hello *ClientHello) {
data := [4]byte{}
payload := acquireBuffer()
defer releaseBuffer(payload)
generateServerHelloHandshakePayload(payload, hello)
// 02 - handshake message type 0x02 (server hello)
// 00 00 76 - 0x76 (118) bytes of server hello data follows
buf.WriteByte(TypeHandshakeServer)
// 00 00 76 - 0x76 (118) bytes of server hello data follows
binary.BigEndian.PutUint32(data[:], uint32(payload.Len()))
buf.Write(data[1:])
payload.WriteTo(buf) //nolint: errcheck
}
func generateServerHelloHandshakePayload(buf *bytes.Buffer, hello *ClientHello) {
// The unusual version number ("3,3" representing TLS 1.2) is due to
// TLS 1.0 being a minor revision of the SSL 3.0 protocol. Therefore
// TLS 1.0 is represented by "3,1", TLS 1.1 is "3,2", and so on.
buf.Write(tls.TLSVersion[:])
buf.Write(emptyRandom[:])
// 20 - 0x20 (32) bytes of session ID follow
// e0 e1 ... fe ff - session ID copied from Client Hello
buf.WriteByte(byte(len(hello.SessionID)))
buf.Write(hello.SessionID)
binary.Write(buf, binary.BigEndian, hello.CipherSuite) //nolint: errcheck
buf.Write(serverHelloSuffix)
scalar := [EllipticCurveLen]byte{}
if _, err := rand.Read(scalar[:]); err != nil {
panic(err)
}
curve, _ := curve25519.X25519(scalar[:], curve25519.Basepoint)
buf.Write(curve)
}
func generateChangeCipherValue(buf *bytes.Buffer) {
buf.WriteByte(tls.TypeChangeCipherSpec)
buf.Write(tls.TLSVersion[:])
binary.Write(buf, binary.BigEndian, uint16(1)) //nolint: errcheck
buf.WriteByte(ChangeCipherValue)
}
// generateNoise writes a single ApplicationData record mimicking the combined
// size of a real TLS 1.3 encrypted server handshake (EncryptedExtensions +
// Certificate chain + CertificateVerify + Finished).
//
// NOTE: Must be exactly ONE ApplicationData record — the Telegram client reads
// ServerHello + CCS + 1 ApplicationData and computes HMAC over all three.
// Multiple records would cause HMAC mismatch and connection failure.
func generateNoise(buf *bytes.Buffer, noise NoiseParams) {
var size int
if noise.Mean > 0 && noise.Jitter > 0 {
// Calibrated: use measured cert chain size ± jitter.
size = noise.Mean - noise.Jitter + rnd.IntN(2*noise.Jitter)
if size < 1000 {
size = 1000
}
} else {
// Legacy fallback: random in 2500-4700 range.
size = 2500 + rnd.IntN(2200)
}
data := make([]byte, size)
if _, err := rand.Read(data); err != nil {
panic(err)
}
tls.WriteRecord(buf, data) //nolint: errcheck
}
@@ -0,0 +1,156 @@
package fake_test
import (
"bytes"
"crypto/hmac"
"crypto/rand"
"crypto/sha256"
"testing"
"github.com/9seconds/mtg/v2/mtglib"
"github.com/9seconds/mtg/v2/mtglib/internal/tls"
"github.com/9seconds/mtg/v2/mtglib/internal/tls/fake"
"github.com/stretchr/testify/suite"
)
type SendServerHelloTestSuite struct {
suite.Suite
hello *fake.ClientHello
buf *bytes.Buffer
secret mtglib.Secret
}
func (suite *SendServerHelloTestSuite) SetupTest() {
suite.hello = &fake.ClientHello{
CipherSuite: 4867,
SessionID: make([]byte, 32),
}
_, err := rand.Read(suite.hello.SessionID)
suite.NoError(err)
_, err = rand.Read(suite.hello.Random[:])
suite.NoError(err)
suite.buf = &bytes.Buffer{}
suite.secret = mtglib.GenerateSecret("google.com")
}
func (suite *SendServerHelloTestSuite) TestRecordStructure() {
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, fake.NoiseParams{})
suite.NoError(err)
var rec bytes.Buffer
recordType, _, err := tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
suite.Equal(byte(tls.TypeHandshake), recordType)
rec.Reset()
recordType, _, err = tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
suite.Equal(byte(tls.TypeChangeCipherSpec), recordType)
rec.Reset()
recordType, length, err := tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
suite.Equal(byte(tls.TypeApplicationData), recordType)
suite.GreaterOrEqual(length, int64(2500))
suite.Empty(suite.buf.Bytes())
}
func (suite *SendServerHelloTestSuite) TestHMAC() {
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, fake.NoiseParams{})
suite.NoError(err)
packet := make([]byte, suite.buf.Len())
copy(packet, suite.buf.Bytes())
random := make([]byte, fake.RandomLen)
copy(random, packet[fake.RandomOffset:])
copy(packet[fake.RandomOffset:], make([]byte, fake.RandomLen))
mac := hmac.New(sha256.New, suite.secret.Key[:])
mac.Write(suite.hello.Random[:])
mac.Write(packet)
suite.Equal(random, mac.Sum(nil))
}
func (suite *SendServerHelloTestSuite) TestHandshakePayload() {
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, fake.NoiseParams{})
suite.NoError(err)
packet := suite.buf.Bytes()
// TLS record header: type(1) + version(2) + length(2)
suite.Equal(byte(tls.TypeHandshake), packet[0])
suite.Equal([]byte{3, 3}, packet[1:3])
// Handshake header: type(1) + uint24_length(3)
suite.Equal(byte(fake.TypeHandshakeServer), packet[5])
// ServerHello version
suite.Equal([]byte{3, 3}, packet[9:11])
// Session ID
sessionIDOffset := fake.RandomOffset + fake.RandomLen
suite.Equal(byte(len(suite.hello.SessionID)), packet[sessionIDOffset])
suite.Equal(suite.hello.SessionID, packet[sessionIDOffset+1:sessionIDOffset+1+len(suite.hello.SessionID)])
}
func (suite *SendServerHelloTestSuite) TestChangeCipherSpec() {
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, fake.NoiseParams{})
suite.NoError(err)
// Skip first record
var rec bytes.Buffer
_, _, err = tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
// Read ChangeCipherSpec record
rec.Reset()
recordType, length, err := tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
suite.Equal(byte(tls.TypeChangeCipherSpec), recordType)
suite.Equal(int64(1), length)
suite.Equal([]byte{fake.ChangeCipherValue}, rec.Bytes())
}
func (suite *SendServerHelloTestSuite) TestCalibratedNoiseSize() {
noise := fake.NoiseParams{Mean: 6480, Jitter: 100}
err := fake.SendServerHello(suite.buf, suite.secret.Key[:], suite.hello, noise)
suite.NoError(err)
var rec bytes.Buffer
// Skip ServerHello
_, _, err = tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
// Skip ChangeCipherSpec
rec.Reset()
_, _, err = tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
// Read noise ApplicationData
rec.Reset()
recordType, length, err := tls.ReadRecord(suite.buf, &rec)
suite.NoError(err)
suite.Equal(byte(tls.TypeApplicationData), recordType)
// Should be within mean ± jitter range.
suite.GreaterOrEqual(length, int64(noise.Mean-noise.Jitter))
suite.LessOrEqual(length, int64(noise.Mean+noise.Jitter))
}
func TestSendServerHello(t *testing.T) {
t.Parallel()
suite.Run(t, &SendServerHelloTestSuite{})
}
@@ -0,0 +1,8 @@
{
"time": 1617181365,
"random": "XvCPc3aAbHbhRLv0kUmy6BfPZOGvsused5/HNsKXEPs=",
"sessionId": "St2BZ2uHMFn3B2trD1jfdtpjoJOOg6JBeLhFcyCMCq4=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "FgMBAgABAAH8AwNe8I9zdoBsduFEu/SRSbLoF89k4a+y6x53n8c2wpcQ+yBK3YFna4cwWfcHa2sPWN922mOgk46DokF4uEVzIIAKrgA0EwMTARMCwCzAK8AkwCPACsAJzKnAMMAvwCjAJ8AUwBPMqACdAJwAPQA8ADUAL8AIwBIACgEAAX//AQABAAAAABsAGQAAFnN0b3JhZ2UuZ29vZ2xlYXBpcy5jb20AFwAAAA0AGAAWBAMIBAQBBQMCAwgFCAUFAQgGBgECAQAFAAUA0AAAADN0AAAAEgAAABAAMAAuAmgyBWgyLTE2BWgyLTE1BWgyLTE0CHNwZHkvMy4xBnNwZHkvMwhodHRwLzEuMQALAAIBAAAzACYAJAAdACAH/ugvH0kSUgAuwslL3UfZA3JTUfSiwrAhR6VWd2wvIgAtAAIBAQArAAkIAwQDAwMCAwEACgAKAAgAHQAXABgAGQAVAKEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="
}
@@ -0,0 +1,8 @@
{
"time": 1617181365,
"random": "XvCPc3aAbHbhRLv0kUmy6BfPZOGvsused5/HNsKXEPs=",
"sessionId": "St2BZ2uHMFn3B2trD1jfdtpjoJOOg6JBeLhFcyCMCq4=",
"host": "storage.googleapis.com",
"cipherSuite": 4867,
"full": "FgMBAgABAAH8AwNe8I9zdoBsduFEu/SRSbLoF89k4a+y6x53n8c2wpcQ+yBK3YFna4cwWfcHa2sPWN922mOgk46DokF4uEVzIIwKrgA0EwMTARMCwCzAK8AkwCPACsAJzKnAMMAvwCjAJ8AUwBPMqACdAJwAPQA8ADUAL8AIwBIACgEAAX//AQABAAAAABsAGQAAFnN0b3JhZ2UuZ29vZ2xlYXBpcy5jb20AFwAAAA0AGAAWBAMIBAQBBQMCAwgFCAUFAQgGBgECAQAFAAUBAAAAADN0AAAAEgAAABAAMAAuAmgyBWgyLTE2BWgyLTE1BWgyLTE0CHNwZHkvMy4xBnNwZHkvMwhodHRwLzEuMQALAAIBAAAzACYAJAAdACAH/ugvH0kSUgAuwslL3UfZA3JTUfSiwrAhR6VWd2wvIgAtAAIBAQArAAkIAwQDAwMCAwEACgAKAAgAHQAXABgAGQAVAKEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="
}

Some files were not shown because too many files have changed in this diff Show More