ScuroNeko/mtg
1
0
Fork 0
mirror of https://github.com/ScuroNeko/mtg.git synced 2026-08-31 18:04:03 +03:00
Code Issues Packages Projects Releases Wiki Activity

Update README

Browse Source
This commit is contained in:
9seconds
2021-04-09 13:30:04 +03:00
parent 43f5d2131e
commit 4f048a3b82
1 changed files with 275 additions and 218 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+275 -218
View File
@@ -1,14 +1,16 @@
# mtg # mtg
Bullshit-free MTPROTO proxy for Telegram Highly-opionated (ex-bullshit-free) MTPROTO proxy for
[Telegram](https://telegram.org/).
[![Build Status](https://travis-ci.org/9seconds/mtg.svg?branch=master)](https://travis-ci.org/9seconds/mtg) [![CI](https://github.com/9seconds/mtg/actions/workflows/ci.yaml/badge.svg?branch=master)](https://github.com/9seconds/mtg/actions/workflows/ci.yaml)
[![Go Report Card](https://goreportcard.com/badge/github.com/9seconds/mtg)](https://goreportcard.com/report/github.com/9seconds/mtg) [![codecov](https://codecov.io/gh/9seconds/mtg/branch/master/graph/badge.svg?token=JfdDyGVpT4)](https://codecov.io/gh/9seconds/mtg)
[![Docker Build Status](https://img.shields.io/docker/build/nineseconds/mtg.svg)](https://hub.docker.com/r/nineseconds/mtg/) [![Go Reference](https://pkg.go.dev/badge/github.com/9seconds/mtg.svg)](https://pkg.go.dev/github.com/9seconds/mtg)
**Please see a guide on upgrading to 1.0 at the end of this README.** **If you use v1.0 or upgrade broke you proxy, please read the chapter
[Version 2](#version-2)**
# Rationale ## Rationale
There are several available proxies for Telegram MTPROTO available. Here There are several available proxies for Telegram MTPROTO available. Here
are the most notable: are the most notable:
@@ -17,267 +19,322 @@ are the most notable:
* [Python](https://github.com/alexbers/mtprotoproxy) * [Python](https://github.com/alexbers/mtprotoproxy)
* [Erlang](https://github.com/seriyps/mtproto_proxy) * [Erlang](https://github.com/seriyps/mtproto_proxy)
Almost all of them follow the way how official proxy was built. This You can use any of these. They work great and all implementations have
includes support of multiple secrets, support of promoted channels, etc. feature parity now. This includes support of adtag, replay attack
protection, domain fronting, faketls, and so on. mtg has a similar
goal: to give a possibility to connect to Telegram in a restricted,
censored environment. But it does it slightly differently in details
that probably matter.
mtg is an implementation in golang which is intended to be: * **Resource-efficient**
It has to be resource-efficient. It does not mean that you will see
the smallest memory usage. It means that it will try to use allocated
resources in zero-waste mode, reusing as much memory as possible and
so on.
* **Lightweight**
It has to consume as few resources as possible but not by losing
maintainability.
* **Easily deployable** * **Easily deployable**
I strongly believe that Telegram proxies should follow the way of I strongly believe that Telegram proxies should follow the way of
ShadowSocks: promoted channels is a strange way of doing business [ShadowSocks](https://shadowsocks.org): promoted channels is a strange
I suppose. I think the only viable way is to have a proxy with way of doing business I suppose. I think the only viable way is to
minimum configuration which should work everywhere. have a proxy that can be restored anywhere easily.
* **A single secret** * **A single secret**
I think that multiple secrets solve no problems and just complexify
I think that multiple secrets solve no problems and just complex
software. I also believe that in the case of throwout proxies, this software. I also believe that in the case of throwout proxies, this
feature is a useless luxury. the feature is a useless luxury.
* **Minimum docker image size**
Official image is less than 3.5 megabytes. Literally. * **No adtag support**
Please read [Version 2](#version-2) chapter.
* **No management WebUI** * **No management WebUI**
This is an implementation of a simple lightweight proxy. I won't do that. This is an implementation of a simple lightweight proxy. I won't do that.
This proxy supports 2 modes of work: direct connection to Telegram and * **Proxy chaining**
promoted channel mode. If you do not need promoted channels, I would
recommend you to go with direct mode: this way is more robust.
To run a proxy in direct mode, all you need to do is just provide a mtg has the support of [SOCKS5](https://en.wikipedia.org/wiki/SOCKS)
secret. If you do not provide ADTag as a second parameter, promoted proxies. So, in theory, you can run this proxy as a frontend
channels mode won't be activated. and route traffic via [v2ray](https://www.v2ray.com/),
[Gost](https://docs.ginuerzh.xyz/gost/),
[Trojan](https://trojan-gfw.github.io/trojan/), or any other project
you like.
To get promoted channel, please contact * **Native blocklist support**
[@MTProxybot](https://t.me/MTProxybot) and provide generated adtag as a
second parameter.
Previously, this was delegated to the [FireHOL](https://firehol.org/)
project or similar ones which track attacks and publish a list of
potentially dangerous IPs. mtg has native support of such blocklists.
# Source code organization * **Can be used as a library**
There are 2 main branches: mtg v2 was redesigned in a way so it can be embedded into your
software (written in Golang) with a minimum effort + you can replace
some parts with those you want.
1. `master` branch contains potentially unstable features ### Version 2
2. `stable` branch contains stable version. Usually you want to use this branch.
# How to build If you use version 1.x before, you are probably noticed some major
backward non-compatible details:
1. Configuration file
2. Removed support of adtag
For the configuration file, please check out the full example in this
repository. It has a lot of comments and most of the options are
optional. We do have only `secret` and `bind-to` sections mandatory.
Other sections in the example configuration file are filled with default
values.
Adtag support was removed completely. This was done to debloat mtg and
keep it simple and obvious. Hopefully, this goal is achieved and the
source code is clean and straightforward enough.
I always was quite skeptical about adtag. In my POV, a proxy as a fat
big connectivity point for hundreds of clients is an illusion. If you
work in a censored environment, the first thing that authority does is
IP blocking. For us, it means, those big proxies that can benefit from
having a pinned channel are going to be blocked in a minute.
Proxy has to be intimate. It has to be shared within a small group as
a family or maybe your college friends. It has to have a small number
of connections and never publicly announced its presence. It has to fly
under the radar. If the proxy is detected, you need to be able to give
a rebirth on a new IP address as soon as possible. I do no think that
having some special channel for such a use case makes any sense.
But other details like replay attack protection, domain fronting,
accurate FakeTLS implementation, IP blacklisting, and proxy
chaining matter here. If you work in censored perimeter like
[GFW](https://en.wikipedia.org/wiki/Great_Firewall)-protected
country, you probably want to have an MTPROTO proxy as
a frontend that transports traffic via cloaked tunnels
made by [Trojan](https://trojan-gfw.github.io/trojan/),
[Shadowsocks](https://shadowsocks.org), [v2ray](https://www.v2ray.com/),
or [Gost](https://docs.ginuerzh.xyz/gost/). That's why you have to have
the support of chaining as a first-class citizen.
Yes, this is possible and doable with optional adtag support. But the
truth is that the MTPROTO proxy for Telegram is just a thing that either
work as a normal client (direct mode) or doing some RPC calls in [TL
language](https://core.telegram.org/mtproto/TL) (adtag support). I
understand the intention of the developers and I understand that they
were under high pressure fighting with [RKN](https://rkn.gov.ru/) and
doing TON after that. Nothing is ideal. But for the proxy, it means that
source code is full of complex non-trivial code which is required only
to support a feature that we barely need.
So, to have a reasonable MTPROTO proxy, adtag support was removed. This
is a rare chance in my career where software v2 debloats a previous
version. It feels so good :)
### Version 1 and 2
I do continue to support both versions 1 and 2. But in a different mode.
Version 1 is now officially in maintenance mode. It means that I won't
make any new features or improvements there. You can consider a feature
freeze there. No bugs are going to be fixed there except for critical
ones. PRs are welcome though. The goal is to keep it working. It will
get some periodical updates like updates to the new Golang version of
dependencies version bump, but that's mostly it.
**If you want to have mtg with _adtag support_, please use version 1**.
Version 2 is going to have all my love, active support, bug fixing, etc.
It is under active development and maintenance.
This project has several main branches
1. [`master`](https://github.com/9seconds/mtg/tree/master) branch
contains a bleeding edge. It may potentially have some features
which will break your source code.
2. [`stable`](https://github.com/9seconds/mtg/tree/stable) branch contains
dumps of a master branch when we consider it 'stable'. This is a
branch you probably want to pick.
3. [`v2`](https://github.com/9seconds/mtg/tree/v2) has a development
of the v2.x version. In theory, it is the same as `master` but this
will change when we have v3.x.
4. [`v1`](https://github.com/9seconds/mtg/tree/v1) has a version 1.x.
## Getting started
### Download a tool
#### Download binaries
Binaries can be downloaded from the release page. Also, you can download
docker image.
For the current version, please download like
```console ```console
make docker pull nineseconds/mtg:2
``` ```
If you want to build for another platform: For version 1:
```console ```console
make crosscompile docker pull nineseconds/mtg:1
``` ```
If you want to build Docker image (called `mtg`): You may also check both [Docker
Hub](https://hub.docker.com/r/nineseconds/mtg/tags) and [Github
Registry](https://github.com/users/9seconds/packages/container/package/mtg).
Please do not choose `latest` or `stable` if you want to avoid
surprises. Always choose some version tag.
Also, if you have `go` installed, you can always download this tool with `go get`:
```console
go get github.com/9seconds/mtg/v2
```
#### Build from sources
```console
git clone https://github.com:9seconds/mtg.git
cd mtg
make static
```
or for the docker image:
```console ```console
make docker make docker
``` ```
# Docker image ### Generate secret
Docker follows the same policy as the source code organization: If you already have a secret in Base64 format or that, which starts with `ee`,
you can skip this chapter. Otherwise:
- `latest` mirrors the master branch
- `stable` mirrors the stable branch
- tags are for tagged releases
```console ```console
docker pull nineseconds/mtg:latest $ mtg generate-secret google.com
7ibaERuTSGPH1RdztfYnN4tnb29nbGUuY29t
``` ```
or
```console ```console
docker pull nineseconds/mtg:stable $ mtg generate-secret --hex google.com
ee473ce5d4958eb5f968c87680a23854a0676f6f676c652e636f6d
``` ```
This secret is a keystone for a proxy and your password for a client.
You need to keep it secured.
We recommend choosing a hostname wisely. Here we have a _google.com_
but in reality, all providers can easily detect that this is not a
Google. Google has a list of networks it officially uses and your IP
address won't probably belong to it. It is a great idea to hide behind
some domain that has some relation to this IP address.
For example, you've bought a VPS from [Digital
Ocean](https://www.digitalocean.com/). Then it might be a good idea to
generate a secret for _digitalocean.com_ then.
### Prepare a configuration file
Please checkout an example configuration file. All options except of
`secret` and `bind-to` are optional. You can safely have this minimal
configuration file:
```toml
secret = "ee473ce5d4958eb5f968c87680a23854a0676f6f676c652e636f6d"
bind-to = "0.0.0.0:443"
```
This is enough to run the whole application. All other
options already have sensible defaults for the app at almost any scale.
Oh, the configuration is done in [TOML format](https://toml.io/en/).
### Run a proxy
Put a binary and a config into your webserver. Just for example,
a binary goes to `/usr/local/bin/mtg` and configuration to `/etc/mtg.toml`.
Now you can create a systemd unit:
```console ```console
docker pull nineseconds/mtg:0.10 $ cat /etc/systemd/system/mtg.service
[Unit]
Description=mtg
[Service]
ExecStart=/usr/local/bin/mtg run /etc/mtg.toml
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
$ sudo systemctl daemon-reload
$ sudo systemctl enable mtg
$ sudo systemctl start mtg
``` ```
# Ansible role or you can run a docker image
You can find unofficial Ansible role for mtg here: https://github.com/rlex/ansible-role-mtg
Also, there is another project on Ansible Galaxy: https://galaxy.ansible.com/ivansible/lin_mtproxy
# Configuration
To run this tool you need to configure as less as possible. Telegram
clients support 3 different secret types:
* Simple - basically, it is just a flow of frames ciphered by AES-CTR stream
cipher.
* Secured - the same stream as simple but with some random noise to prevent
statistical analysis of traffic flow.
* FakeTLS - this mode envelops telegram stream in TLS so it looks (in theory)
the same as any TLS1.3 traffic from DPI point of view.
If you do not have preferences, go with FakeTLS or at least secured.
Simple mode is a little bit naive and traffic flow can be easily
identified as Telegram one.
Unlike the rest of implementation, mtg is quite strict about the
execution mode: if you run a proxy instance with FakeTLS secret, you
can't connect to it with simple or secured clients. You can't connect
to the proxy with secured secret with FakeTLS key. It forces one mode
of working. So, unfortunately, there is no way how to connect to the
deployed proxy with another secret (if you know how to construct and
convert them). But at the same time, old clients can't connect so they
won't expose the type of the service.
First, you need to generate a secret:
```console ```console
$ mtg generate-secret simple docker run -d -v /etc/mtg.toml:/config.toml -p 443:3128 --restart=unless-stopped nineseconds/mtg:2
52a493bdfb90eea55739eabff2d92a14
``` ```
where _443_ is a host port (a port you want to connect to from a
client), and _3128_ is the one you have in your config in the `bind-to`
section.
### Access a proxy
Now you can generate some useful links:
```console ```console
$ mtg generate-secret secured $ mtg access /etc/mtg.toml
ddf05fb7acb549be047a7c585116581418 {
"ipv4": {
"ip": "x.y.z.a",
"port": 3128,
"tg_url": "tg://proxy?...",
"tg_qrcode": "https://api.qrserver.com/v1/create-qr-code?data...",
"tme_url": "https://t.me/proxy?...",
"tme_qrcode": "https://api.qrserver.com/v1/create-qr-code?data..."
},
"secret": {
"hex": "...",
"base64": "..."
}
}
``` ```
```console ## Metrics
$ mtg generate-secret -c google.com tls
ee852380f362a09343efb4690c4e17862e676f6f676c652e636f6d
```
Or, if you prefer docker: Out of the box, mtg works with
[statsd](https://github.com/statsd/statsd) and
[Prometheus](https://prometheus.io/). Please check configuration file
example to get how to set this integration up.
```console Here goes a list of metrics with their types but without a prefix.
$ docker run --rm nineseconds/mtg generate-secret tls -c bing.com
eedf71035a8ed48a623d8e83e66aec4d0562696e672e636f6d
```
## Antireplay cache | Name | Type | Tags | Description |
|-----------------------------|---------|----------------------------------|--------------------------------------------------------------------------------------------|
| client_connections | gauge | `ip_family` | Count of processing client connections. |
| telegram_connections | gauge | `telegram_ip`, `dc` | Count of connections to Telegram servers. |
| domain_fronting_connections | gauge | `ip_family` | Count of connections to fronting domain. |
| telegram_traffic | counter | `telegram_ip`, `dc`, `direction` | Count of bytes, transmitted to/from Telegram. |
| domain_fronting_traffic | counter | `direction` | Count of bytes, transmitted to/from fronting domain. |
| domain_fronting | counter | – | Count of domain fronting events. |
| concurrency_limited | counter | – | Count of events, when client connection was rejected due to concurrency limit. |
| ip_blocklisted | counter | – | Count of events when client connection was rejected because IP was found in the blacklist. |
| replay_attacks | counter | – | Count of detected replay attacks. |
To prevent replay attacks, we have internal storage of first frames Tag meaning:
messages for connected clients. These frames are generated randomly
by design and we have the negligible possibility of duplication
(probability is 1/(2^64)) but it could be quite effective to prevent
replays.
It is possible to disable this cache. To do that, please explicitly set | Name | Values | Description |
its size to 0. |-------------|------------------------|-----------------------------------------------|
| ip_family | ipv4, ipv6 | A version of the IP protocol. |
| dc | | A number of the Telegram DC for a connection. |
## FakeTLS | telegram_ip | | IP address of the Telegram server. |
| direction | to_client, from_client | A direction of the traffic flow. |
If you run this a proxy in faketls mode, this proxy will try to hide
itself cloaking a host provided as a part of the generated secret. It
means that if you cloak google.com then you can curl this proxy and
you'll get a google.com response back.
mtg proxies L3 traffic. In other words, only TCP, without interfering in
TLS, HTTP or any other high-level protocol.
## Environment variables
It is possible to configure this tool using environment variables. You
can configure any flag but not secret or adtag. Here is the list of
supported environment variables:
| Environment variable | Corresponding flags | Default value | Description |
|-------------------------------|------------------------------|-----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `MTG_DEBUG` | `-d`, `--debug` | `false` | Run in debug mode. Usually, you need to run in this mode only if you develop this tool or its maintainer is asking you to provide logs with such verbosity. |
| `MTG_VERBOSE` | `-v`, `--verbose` | `false` | Run in verbose mode. This is way less chatty than debug mode. |
| `MTG_BIND` | `-b`, `--bind` | `0.0.0.0:3128` | Which host/port pair should we bind to (listen on). |
| `MTG_IPV4` | `-4`, `--public-ipv4` | [Autodetect](https://ifconfig.co) | IPv4 address:port of this proxy. This is required if you NAT your proxy or run it in a docker container. In that case, you absolutely need to specify public IPv4 address of the proxy, otherwise either URLs will be broken or proxy could not access Telegram middle proxies. |
| `MTG_IPV6` | `-6`, `--public-ipv6` | [Autodetect](https://ifconfig.co) | IPv6 address:port of this proxy. This is required if you NAT your proxy or run it in a docker container. In that case, you absolutely need to specify public IPv6 address of the proxy, otherwise either URLs will be broken or proxy could not access Telegram middle proxies. |
| `MTG_STATS_BIND` | `-t`, `--stats-bind` | `127.0.0.1:3129` | Which hist:port should we bind the internal statistics HTTP server (Prometheus). |
| `MTG_STATS_NAMESPACE` | `--stats-namespace` | `mtg` | Which namespace should be used for prometheus metrics. |
| `MTG_STATSD_ADDR` | `--statsd-addr` | | host:port of statsd service. No defaults, by default we do not send anything there. |
| `MTG_STATSD_PREFIX` | `--statsd-prefix` | `mtg` | Which bucket prefix we should use. For example, if you set `mtg`, then metric `traffic.ingress` would be send as `mtg.traffic.ingress`. |
| `MTG_STATSD_TAGS_FORMAT` | `--statsd-tags-format` | | Which tags format we should use. By default, we are using default vanilla statsd tags format but if you want to send directly to InfluxDB or Datadog, please specify it there. Possible options are `influxdb` and `datadog`. |
| `MTG_STATSD_TAGS` | `--statsd-tags` | | Which tags should we send to statsd with our metrics. Please specify them as `key=value` pairs. |
| `MTG_BUFFER_WRITE` | `-w`, `--write-buffer` | `32KB` | The size of TCP write buffer in bytes. Write buffer is the buffer for messages which are going from client to Telegram. |
| `MTG_BUFFER_READ` | `-r`, `--read-buffer` | `32KB` | The size of TCP read buffer in bytes. Read buffer is the buffer for messages from Telegram to client. |
| `MTG_ANTIREPLAY_MAXSIZE` | `--anti-replay-max-size` | `128MB` | Max size of antireplay cache. |
| `MTG_CLOAK_PORT` | `--cloak-port` | `443` | Which port we should use to connect to cloaked host in FakeTLS mode. |
| `MTG_MULTIPLEX_PERCONNECTION` | `--multiplex-per-connection` | `50` | How many client connections can share a single Telegram connection in adtag mode |
| `MTG_NTP_SERVERS` | `--ntp-server` | default pool | A list of NTP servers to use. |
| `MTG_PREFER_DIRECT_IP` | `--prefer-ip` | `ipv6` | Which IP protocol to prefer if possible. Works mostly in direct mode. |
Usually you want to modify only read/write buffer sizes. If you feel
that proxy is slow, try to increase both sizes giving more priority to
read buffer.
Unfortunately, MTPROTO proxy protocol does not allow us to use splice
or any other neat tricks how to eliminate the need of copying data into
userspace.
# How to run the tool
Now run the tool:
```console
$ mtg run <secret>
```
How to run the tool with ADTag:
```console
$ mtg run <secret> <adtag>
```
This tool will listen on port 3128 by default with the given secret.
# oneliner to run this proxy
Please ensure that docker is installed. After that just execute
```console
curl -sfL --compressed https://raw.githubusercontent.com/9seconds/mtg/master/run.sh | bash
```
# statsd integration
mtg provides an integration with statsd, you can enable it with command
line interface. To enable it, you have to provide IP address of statsd
service.
Out of the box, mtg supports 2 additional dialects: [InfluxDB](https://www.influxdata.com/blog/getting-started-with-sending-statsd-metrics-to-telegraf-influxdb/)
and [Datadog](https://docs.datadoghq.com/developers/dogstatsd/).
All metrics are gauges. Here is the list of metrics and their meaning:
| Metric name | Unit | Description |
|------------------------|---------|--------------------------------------------|
| `connections` | number | The number of active connections. |
| `telegram_connections` | number | The number of active telegram connections. |
| `crashes` | number | An amount of crashes in client handlers. |
| `traffic.egress` | bytes | Traffic from the start of application. |
| `replay_attacks` | number | The number of prevented replay attacks. |
All metrics are prefixed with given prefix. Default prefix is `mtg`.
Also, metrics provide tags (ipv4/ipv6, dc indexes etc).
# Prometheus integration
[Prometheus](https://prometheus.io) integration comes out of
the box, you do not need to setup anything special.
# Upgrade to 1.0
Version 1.0 breaks compatibility with previous versions so please read
this chapter carefully:
1. mtg now uses subcommands. Please use `mtg run` instead of just
`mtg` to run a proxy.
2. Options which set host and port separately were removed in a
favor of fused `host:port` options.
3. Own stats server was removed. Prometheus endpoint is moved to
default stats endpoint.
4. It is possible to connect to this proxy only with a secret which
was used to run it. So, no backward compatibility of clients.
5. Multiplexing involves connectivity with middle proxies and involves
the most complex code path of this proxy. To avoid potential bugs,
we still recommend using direct mode.