Erlang mtproto proxy

This part of code was extracted from @socksy_bot.

Features

• Promoted channels! See mtproto_proxy_app.src tag option.
• "secure" randomized-packet-size protocol (34-symbol secrets starting with 'dd') to prevent detection by DPI
• Fake-TLS protocol (base64 secrets) - another protocol to prevent DPI detection
• Secure-only mode (only allow connections with 'dd' or fake-tls-base64-secrets). See allowed_protocols option.
• Multiple ports with unique secret and promo tag for each port
• Very high performance - can handle tens of thousands connections! Scales to all CPU cores. 1Gbps, 90k connections on 4-core/8Gb RAM cloud server.
• Supports multiplexing (Many connections Client -> Proxy are wrapped to small amount of connections Proxy -> Telegram Server)
• Protection from replay attacks used to detect proxies in some countries
• Automatic telegram configuration reload (no need for restarts once per day)
• Most of the configuration options can be updated without service restart
• Small codebase compared to official one, code is covered by automated tests
• A lots of metrics could be exported (optional)

How to start - docker

To run with default settings

docker run -d --network=host seriyps/mtproto-proxy

To run on single port with custom port, secret and ad-tag

docker run -d --network=host seriyps/mtproto-proxy -p 443 -s d0d6e111bada5511fcce9584deadbeef -t dcbe8f1493fa4cd9ab300891c0b5b326

or via environment variables

docker run -d --network=host -e MTP_PORT=443 -e MTP_SECRET=d0d6e111bada5511fcce9584deadbeef -e MTP_TAG=dcbe8f1493fa4cd9ab300891c0b5b326 seriyps/mtproto-proxy

Where

• -p 443 / MTP_PORT=… proxy port
• -s d0d6e111bada5511fcce9584deadbeef / MTP_SECRET=… proxy secret (don't append dd! it should be 32 chars long!)
• -t dcbe8f1493fa4cd9ab300891c0b5b326 / MTP_TAG=… ad-tag that you get from @MTProxybot
• -a dd / MTP_DD_ONLY=t only allow "secure" connections (dd-secrets)
• -a tls / MTP_TLS_ONLY=t only allow "fake-TLS" connections (base64 secrets)

It's ok to provide both -a dd -a tls to allow both protocols. If no -a option provided, all protocols will be allowed.

To run with custom config-file

1. Get the code git clone https://github.com/seriyps/mtproto_proxy.git && cd mtproto_proxy/
2. Copy config templates cp config/{vm.args.example,prod-vm.args}; cp config/{sys.config.example,prod-sys.config}
3. Edit configs. See Settings.
4. Build docker build -t mtproto-proxy-erl .
5. Start docker run -d --network=host mtproto-proxy-erl

Installation via docker can work well for small setups (10-20k connections), but for more heavily-loaded setups it's recommended to install proxy directly into your server's OS (see below).

How to start OS-install - quick

You need at least Erlang version 20! Recommended OS is Ubuntu 18.04.

sudo apt install erlang-nox erlang-dev build-essential
git clone https://github.com/seriyps/mtproto_proxy.git
cd mtproto_proxy/
cp config/vm.args.example config/prod-vm.args
cp config/sys.config.example config/prod-sys.config
# configure your port, secret, ad_tag. See [Settings](#settings) below.
nano config/prod-sys.config
make && sudo make install
sudo systemctl enable mtproto-proxy
sudo systemctl start mtproto-proxy

How to start OS-install - detailed

Install deps (ubuntu 18.04)

sudo apt install erlang-nox erlang-dev build-essential

You need Erlang version 20 or higher! If your version is older, please, check Erlang solutions esl-erlang package or use kerl.

Get the code:

git clone https://github.com/seriyps/mtproto_proxy.git
cd mtproto_proxy/

Create config file

see Settings.

Build and install

make && sudo make install

This will:

• install proxy into /opt/mtp_proxy
• create a system user
• install systemd service
• create a directory for logs in /var/log/mtproto-proxy
• Configure ulimit of max open files and CAP_NET_BIND_SERVICE by systemd

Try to start in foreground mode

This step is optional, but it can be usefull to test if everything works as expected

./start.sh

try to run ./start.sh -h to learn some useful options.

Start in background and enable start on system start-up

sudo systemctl enable mtproto-proxy
sudo systemctl start mtproto-proxy

Done! Proxy is up and ready to serve now!

Stop / uninstall

Stop:

sudo systemctl stop mtproto-proxy

Uninstall:

sudo systemctl stop mtproto-proxy
sudo systemctl disable mtproto-proxy
sudo make uninstall

Logs can be found at

/var/log/mtproto-proxy/application.log

Settings

All available documented configuration options could be found in src/mtproto_proxy.app.src. Do not edit this file!

To change configuration, edit config/prod-sys.config:

Comments in this file start with %%. Default port is 1443 and default secret is d0d6e111bada5511fcce9584deadbeef.

Secret key and proxy URLs will be printed on start.

The easiest way to update config right now is to edit config/prod-sys.config and then re-install proxy by

sudo make uninstall && make && sudo make install

There are other ways as well. It's even possible to update configuration options without service restart / without downtime, but it's a bit trickier.

Change default port / secret / ad tag

To change default settings, change mtproto_proxy section of prod-sys.config as:

 {mtproto_proxy,
  %% see src/mtproto_proxy.app.src for examples.
  [
   {ports,
    [#{name => mtp_handler1,
       listen_ip => "0.0.0.0",
       port => 1443,
       secret => <<"d0d6e111bada5511fcce9584deadbeef">>,
       tag => <<"dcbe8f1493fa4cd9ab300891c0b5b326">>}
    ]}
   ]},

 {lager,
<...>

(so, remove %%s) and replace port / secret / tag with yours.

Listen on multiple ports / IPs

You can start proxy on many IP addresses or ports with different secrets/ad tags. To do so, just add more configs to ports section, separated by comma, eg:

 {mtproto_proxy,
  %% see src/mtproto_proxy.app.src for examples.
  [
   {ports,
    [#{name => mtp_handler_1,
       listen_ip => "0.0.0.0",
       port => 1443,
       secret => <<"d0d6e111bada5511fcce9584deadbeef">>,
       tag => <<"dcbe8f1493fa4cd9ab300891c0b5b326">>},
     #{name => mtp_handler_2,
       listen_ip => "0.0.0.0",
       port => 2443,
       secret => <<"100000000000000000000000000000001">>,
       tag => <<"cf8e6baff125ed5f661a761e69567711">>}
    ]}
   ]},

 {lager,
<...>

Each section should have unique name!

Only allow connections with 'dd'-secrets

This protocol uses randomized packet sizes, so it's more difficult to detect on DPI by packet sizes. It might be useful in Iran, where proxies are detected by DPI. You should disable all protocols other than mtp_secure by providing allowed_protocols option:

  {mtproto_proxy,
   [
    {allowed_protocols, [mtp_secure]},
    {ports,
     [#{name => mtp_handler_1,
      <..>

Only allow fake-TLS connections with base64-secrets

Another censorship circumvention technique. MTPRoto proxy protocol pretends to be HTTPS web traffic (technically speaking, TLSv1.3 + HTTP/2). It's possible to only allow connections with this protocol by changing allowed_protocols to be list with only mtp_fake_tls:

  {mtproto_proxy,
   [
    {allowed_protocols, [mtp_fake_tls]},
    {ports,
     [#{name => mtp_handler_1,
      <..>

IPv6

Currently proxy only supports client connections via IPv6, but can only connect to Telegram servers using IPv4.

To enable IPv6, you should put IPv6 address in listen_ip config key. If you want proxy to accept clients on the same port with both IPv4 and IPv6, you should have 2 ports sections with the same port, secret and tag, but with different names and different listen_ip (one v4 and one v6):

 {mtproto_proxy,
  %% see src/mtproto_proxy.app.src for examples.
  [
   {ports,
    [#{name => mtp_handler_all_ipv4,
       listen_ip => "0.0.0.0",  % IPv4 address, eg 203.0.113.1
       port => 1443,
       secret => <<"d0d6e111bada5511fcce9584deadbeef">>,
       tag => <<"dcbe8f1493fa4cd9ab300891c0b5b326">>},
     #{name => mtp_handler_all_ipv6,
       listen_ip => "::",  % IPv6 address, eg "2001:db8:85a3::8a2e:370:7334"
       port => 1443,
       secret => <<"d0d6e111bada5511fcce9584deadbeef">>,
       tag => <<"dcbe8f1493fa4cd9ab300891c0b5b326">>}
    ]}
   ]},

 {lager,
<...>

Tune resource consumption

If your server have low amount of RAM, try to set

{upstream_socket_buffer_size, 5120},
{downstream_socket_buffer_size, 51200},
{replay_check_session_storage, off},

this may make proxy slower, it can start to consume bit more CPU, will be vulnerable to replay attacks, but will use less RAM.

If your server have lots of RAM, you can make it faster (users will get higher uppload/download speed), it will use less CPU and will be better protected from replay attacks, but will use more RAM:

{max_connections, 128000},
{upstream_socket_buffer_size, 20480},
{downstream_socket_buffer_size, 512000},
{replay_check_session_storage, on},
{replay_check_session_storage_opts,
  #{max_memory_mb => 2048,
    max_age_minutes => 1440}},

Also, for highload setups it's recommended to increase sysctl parameters:

sudo sysctl net.ipv4.tcp_max_orphans=128000
sudo sysctl 'net.ipv4.tcp_mem=179200 256000 384000'

Values for tcp_mem are in pages. Size of one page can be found by getconf PAGESIZE and is most likely 4kb.

If you have installed proxy via Docker or use some NAT firewall settings, you may want to increase netfilter conntrack limits to be at least the max number of connections you expect:

sudo sysctl net.netfilter.nf_conntrack_max=128000

Helpers

Number of connections

/opt/mtp_proxy/bin/mtp_proxy eval 'lists:sum([proplists:get_value(all_connections, L) || {_, L} <- ranch:info()]).'