CERBERUS/mtproto_proxy
1
0
Fork 0
mirror of https://github.com/seriyps/mtproto_proxy.git synced 2026-05-13 08:46:46 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: split-mode setup guide, architecture diagrams, cert script, build

Browse source 
README:
- New 'Split-mode setup' section: motivation, firewall rules, step-by-step
  instructions for both VPN tunnel and TLS distribution options
- Split-mode bullet added to Features list
- Notes on DPI-resistant tunnels (Shadowsocks, VLESS/XRay, Hysteria2) for
  Russian deployment; standard VPN protocols (WireGuard, OpenVPN) may be blocked
- Install instructions updated to use `make init-config` (copies templates,
  auto-detects public IP) instead of manual cp; ROLE= documented throughout
- Split-mode Step 4 uses `make ROLE=back/front` so template-change detection
  works correctly after `git pull`

Makefile:
- ROLE ?= both variable selects config templates (both/front/back)
- Config prereq rules use $(SYS_CONFIG_SRC) / $(VM_ARGS_SRC) based on ROLE
- New `init-config` target: force-copies templates, auto-detects public IP,
  prints edit reminder; replaces manual cp in install workflow

scripts/gen_dist_certs.sh:
- Two-step workflow: `init <dir>` on back server (CA + back cert),
  `add-node <dir> <name>` per front server (cert signed by existing CA)
- Generates per-node ssl_dist.<name>.conf with paths substituted (no
  NODE_NAME placeholder to edit manually)
- ssl_dist.<name>.conf is now used directly (no rename to ssl_dist.conf);
  vm.args examples and README updated to match

config/vm.args.{front,back}.example:
- -ssl_dist_optfile points to role-specific filename (ssl_dist.front.conf /
  ssl_dist.back.conf) so cert files can be copied as-is without renaming

AGENTS.md:
- Role-overview Mermaid flowchart showing front/back/both process split
- Data-plane section replaced with links to doc/ (no duplication)
- Supervision tree, key interactions, split-mode config keys updated

doc/handler-downstream-flow.md, doc/migration-flow.md:
- Mermaid box grouping to visually separate FRONT and BACK node participants
- erpc:call reference corrected (was rpc:call)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Sergey Prokhorov 2026-04-11 23:33:44 +02:00
parent 42d86517a4
commit 121d8b7413
No known key found for this signature in database
GPG key ID: 1C570244E4EF3337
8 changed files with 487 additions and 62 deletions
Download patch file Download diff file Expand all files Collapse all files

118
AGENTS.md
View file

@ -29,49 +29,78 @@ Dockerfile Docker image build
| `mtp_secure` | "Secure" randomized-packet-size protocol |
| `mtp_dc_pool` / `mtp_down_conn` | Pooled/multiplexed connections to Telegram DCs |
| `mtp_rpc` | RPC framing protocol between proxy and Telegram |
| `mtp_config` | Periodically fetches Telegram DC configuration |
| `mtp_config` | Periodically fetches Telegram DC configuration; in split mode exposes `backend_node/0` and remote-aware `get_downstream_pool/1` / `get_default_dc/0` |
| `mtp_policy` / `mtp_policy_table` | Connection limit, blacklist, and whitelist rules |
| `mtp_codec` / `mtp_aes_cbc` | Codec pipeline (MTProto framing + AES-CBC encryption) |
| `mtp_abridged` / `mtp_full` / `mtp_intermediate` | MTProto transport codec variants |
| `mtp_metric` | Metrics/telemetry |
| `mtp_metric` | Metrics/telemetry; `passive_metrics/0` is role-aware |
| `mtp_session_storage` | Replay-attack protection (session deduplication) |
| `mtproto_proxy_sup` | Root supervisor; calls `children(Role)` — children differ by `node_role` |
| `mtproto_proxy_app` | OTP application callback; `start/2` and `config_change/3` are role-gated |
### Process architecture
**Role overview** — what starts in each `node_role`:
```mermaid
flowchart LR
subgraph BOTH["node_role=both (single server, default)"]
direction TB
F0["Ranch listeners\nmtp_handler\nmtp_session_storage\nmtp_policy_*"]
B0["mtp_config\nmtp_dc_pool\nmtp_down_conn"]
end
subgraph SPLIT["Split mode (two servers)"]
direction TB
subgraph FRONT["node_role=front (domestic server)"]
F1["Ranch listeners\nmtp_handler\nmtp_session_storage\nmtp_policy_*"]
end
subgraph BACK["node_role=back (foreign server)"]
B1["mtp_config\nmtp_dc_pool\nmtp_down_conn"]
end
FRONT -- "Erlang distribution\n(TLS or VPN tunnel)" --> BACK
end
```
```
OTP supervision tree
────────────────────────────────────────────────────────────────────
mtproto_proxy_sup (one_for_one)
├── mtp_config (gen_server, singleton)
├── mtp_session_storage (gen_server, singleton)
├── mtp_dc_pool_sup (supervisor, simple_one_for_one)
│ └── mtp_dc_pool (gen_server, one per DC id, permanent)
├── mtp_down_conn_sup (supervisor, simple_one_for_one)
│ └── mtp_down_conn (gen_server, one per Telegram TCP conn, temporary)
└── Ranch listeners (one per configured port: mtp_ipv4, mtp_ipv6, …)
└── mtp_handler (gen_server, one per client TCP conn, transient)
The supervisor is role-parameterised via the `node_role` config key
(`front | back | both`, default `both`). Each role starts a different
subset of children:
node_role=both (default — single server)
├── mtp_config (gen_server, singleton)
├── mtp_session_storage (gen_server, singleton)
├── mtp_dc_pool_sup (supervisor, simple_one_for_one)
│ └── mtp_dc_pool (gen_server, one per DC id, permanent)
├── mtp_down_conn_sup (supervisor, simple_one_for_one)
│ └── mtp_down_conn (gen_server, one per Telegram TCP conn, temporary)
└── Ranch listeners (one per configured port: mtp_ipv4, mtp_ipv6, …)
└── mtp_handler (gen_server, one per client TCP conn, transient)
Data-plane message flow (one client connection)
────────────────────────────────────────────────────────────────────
node_role=front (domestic server — accepts Telegram clients)
├── mtp_session_storage
├── mtp_policy_table
├── mtp_policy_counter
└── Ranch listeners → mtp_handler
node_role=back (foreign server — connects to Telegram DCs)
├── mtp_config
├── mtp_dc_pool_sup → mtp_dc_pool
└── mtp_down_conn_sup → mtp_down_conn
In split mode, the front node holds `back_node` in its config and
addresses back-node processes as `{RegisteredName, BackNode}`.
Multiple front nodes can share one back node.
```
**Data-plane message flow** — see [`doc/handler-downstream-flow.md`](doc/handler-downstream-flow.md)
for the full sequence diagram (pool lookup, steady-state data exchange, connection release) and
[`doc/migration-flow.md`](doc/migration-flow.md) for transparent DC connection rotation.
Both diagrams show the front/back node boundary. In `both` mode all processes share the same
node and there is no distribution overhead.
Telegram client Telegram server
│ │
│ TCP (fake-TLS / obfuscated / secure) │
▼ ▼
┌─────────────┐ gen_server:call({send,Data}) ┌──────────────┐ raw TCP ┌──────────────┐
│ mtp_handler │ ──────────────────────────────► │ mtp_down_conn│ ────────────► │ Telegram DC │
│ (upstream) │ │ (downstream) │ ◄──────────── │ (middle srv) │
│ │ ◄────────────────────────────── │ │ └──────────────┘
└──────┬──────┘ gen_server:cast({proxy_ans}) └──────────────┘
│
│ on first data: mtp_config:get_downstream_safe/2 → picks (pool, down_conn)
│ on disconnect: cast({return, self()}) → releases slot
▼
┌─────────────┐
│ mtp_dc_pool │ — spawns mtp_down_conn via mtp_down_conn_sup when pool is empty
│ (per DC) │
└─────────────┘
> **Naming note:** the terms "upstream" and "downstream" in the current code are the
> opposite of what one might expect:
@ -80,12 +109,17 @@ Data-plane message flow (one client connection)
> This will be renamed in a future refactor.
Key interactions
────────────────────────────────────────────────────────────────────
**Key interactions:**
```
mtp_handler → mtp_config : get_downstream_safe/2 — resolves DC id to
a (pool_pid, down_conn_pid) pair on first
upstream data packet
mtp_handler → mtp_down_conn : send/2 (sync call) — forward client data
upstream data packet.
In split mode returns {PoolName, BackNode};
uses erpc:call to check pool existence on
the back node.
mtp_handler → mtp_down_conn : send/2 (sync call) — forward client data;
in split mode this is a cross-node gen_server call
mtp_down_conn → mtp_handler : cast {proxy_ans, …} — forward Telegram reply
mtp_down_conn → mtp_handler : cast {close_ext, …} — Telegram closed stream
mtp_handler → mtp_dc_pool : return/2 (cast) — release slot on disconnect
@ -147,9 +181,9 @@ There are three kinds of tests, each with a clear home:
|------|-------|-------------|
| **EUnit** (unit) | `src/*.erl`, `-ifdef(TEST)` blocks | Pure functions with no I/O: codec encode/decode round-trips, packet parsing helpers, crypto primitives |
| **PropEr** (property-based) | `test/prop_mtp_<module>.erl` | Codec/parser properties that should hold for *arbitrary* inputs — e.g. encode→decode identity, parser accepts all valid inputs, parser never crashes on random bytes |
| **Common Test** (integration) | `test/single_dc_SUITE.erl` | End-to-end behaviour involving a real listener + fake DC: protocol negotiation, policy enforcement, error handling visible at the TCP level (alerts sent, connections closed), domain fronting, replay protection |
| **Common Test** (integration) | `test/single_dc_SUITE.erl`, `test/split_dc_SUITE.erl` | End-to-end behaviour involving a real listener + fake DC: protocol negotiation, policy enforcement, error handling visible at the TCP level (alerts sent, connections closed), domain fronting, replay protection. `split_dc_SUITE` tests the same paths in split mode (front/back on separate `peer` nodes). |
**Rule of thumb:** if the behaviour is observable only over a TCP socket or requires a running application, it belongs in `single_dc_SUITE`. If it is a property of a pure function, add a PropEr property in the matching `prop_mtp_<module>.erl`. If it is a targeted unit case for a specific input, use EUnit.
**Rule of thumb:** if the behaviour is observable only over a TCP socket or requires a running application, it belongs in `single_dc_SUITE`. If it requires two nodes communicating over Erlang distribution, it belongs in `split_dc_SUITE`. If it is a property of a pure function, add a PropEr property in the matching `prop_mtp_<module>.erl`. If it is a targeted unit case for a specific input, use EUnit.
**What changes need new tests:**
@ -199,6 +233,18 @@ Workflow:
- All configuration options are documented in `src/mtproto_proxy.app.src`.
- Config can be reloaded without restart: `make update-sysconfig && systemctl reload mtproto-proxy`.
### Split-mode config keys
| Key | Node | Meaning |
|-----|------|---------|
| `node_role` | both | `front \| back \| both` (default `both` — single-server mode) |
| `back_node` | front only | Atom name of the back node, e.g. `'back@10.0.0.2'` |
| `external_ip` | back | Public IP of the back server (used in the RPC handshake AES key) |
`mtp_config` is **not started on a front node** — never call `mtp_config:status()` or any
`mtp_config` function from code that runs on a front node. `get_downstream_safe/2` is the
correct entry point; it is role-aware and dispatches remotely when needed.
## Debugging
### Enabling debug logs for a single module at runtime

31
Makefile
View file

@ -6,6 +6,23 @@ EPMD_SERVICE:=$(DESTDIR)/etc/systemd/system/epmd.service
LOGDIR:=$(DESTDIR)/var/log/mtproto-proxy
USER:=mtproto-proxy
# ROLE selects which config templates are used.
# Values: both (default, single server), front (domestic), back (foreign).
# For split mode: run `make init-config ROLE=front` / `make init-config ROLE=back`
# on each server, edit the resulting config files, then run `make ROLE=front` etc.
ROLE ?= both
ifeq ($(ROLE),front)
SYS_CONFIG_SRC := config/sys.config.front.example
VM_ARGS_SRC := config/vm.args.front.example
else ifeq ($(ROLE),back)
SYS_CONFIG_SRC := config/sys.config.back.example
VM_ARGS_SRC := config/vm.args.back.example
else
SYS_CONFIG_SRC := config/sys.config.example
VM_ARGS_SRC := config/vm.args.example
endif
all: config/prod-sys.config config/prod-vm.args
$(REBAR3) as prod release
@ -19,15 +36,25 @@ test:
$(REBAR3) dialyzer
$(REBAR3) cover -v
config/prod-sys.config: config/sys.config.example
config/prod-sys.config: $(SYS_CONFIG_SRC)
[ -f $@ ] && diff -u $@ $^ || true
cp -i -b $^ $@
config/prod-vm.args: config/vm.args.example
config/prod-vm.args: $(VM_ARGS_SRC)
[ -f $@ ] && diff -u $@ $^ || true
cp -i -b $^ $@
@IP=$(shell curl -s -4 -m 10 http://ip.seriyps.com || curl -s -4 -m 10 https://digitalresistance.dog/myIp) \
&& sed -i s/@0\.0\.0\.0/@$${IP}/ $@
.PHONY: init-config
init-config:
cp $(SYS_CONFIG_SRC) config/prod-sys.config
cp $(VM_ARGS_SRC) config/prod-vm.args
@IP=$$(curl -s -4 -m 10 http://ip.seriyps.com || curl -s -4 -m 10 https://digitalresistance.dog/myIp) \
&& sed -i s/@0\.0\.0\.0/@$${IP}/ config/prod-vm.args; true
@echo ""
@echo "Config created from ROLE=$(ROLE) templates."
@echo "Edit config/prod-sys.config and config/prod-vm.args, then run: make [ROLE=$(ROLE)]"
user:
sudo useradd -r $(USER) || true

190
README.md
View file

@ -37,6 +37,10 @@ Features
* Automatic telegram configuration reload (no need for restarts once per day)
* IPv6 for client connections
* All configuration options can be updated without service restart
* Split-mode setup: run the client-facing part (front) on a domestic server and
the Telegram-facing part (back) on a foreign server, connected via Erlang distribution
(TLS or a censorship-resistant tunnel). Bypasses ISP blocks that target direct
domestic→foreign connections. Multiple front servers can share one back server.
* Small codebase compared to official one, code is covered by automated tests
* A lots of metrics could be exported (optional)
@ -94,7 +98,7 @@ It's ok to provide both `-a dd -a tls` to allow both protocols. If no `-a` optio
### To run with custom config-file
1. Get the code `git clone https://github.com/seriyps/mtproto_proxy.git && cd mtproto_proxy/`
2. Copy config templates `cp config/{vm.args.example,prod-vm.args}; cp config/{sys.config.example,prod-sys.config}`
2. Copy config templates `make init-config` (or manually: `cp config/{vm.args.example,prod-vm.args}; cp config/{sys.config.example,prod-sys.config}`)
3. Edit configs. See [Settings](#settings).
4. Build `docker build -t mtproto-proxy-erl .`
5. Start `docker run -d --network=host mtproto-proxy-erl`
@ -112,8 +116,7 @@ You need at least Erlang version 25! Recommended OS is Ubuntu 24.04.
sudo apt install erlang-nox erlang-dev build-essential
git clone https://github.com/seriyps/mtproto_proxy.git
cd mtproto_proxy/
cp config/vm.args.example config/prod-vm.args
cp config/sys.config.example config/prod-sys.config
make init-config # copies templates and auto-detects your server's IP
# configure your port, secret, ad_tag. See [Settings](#settings) below.
nano config/prod-sys.config
make && sudo make install
@ -158,7 +161,12 @@ cd mtproto_proxy/
### Create config file
see [Settings](#settings).
```bash
make init-config # copies sys.config.example → prod-sys.config and vm.args.example → prod-vm.args
# also auto-detects your server's public IP
```
Edit `config/prod-sys.config` — see [Settings](#settings) for all options.
### Build and install
@ -810,3 +818,177 @@ Number of connections
```bash
/opt/mtp_proxy/bin/mtp_proxy eval 'lists:sum([maps:get(all_connections, L) || {_, L} <- maps:to_list(ranch:info())]).'
```
Split-mode setup (front + back)
--------------------------------
### Why split mode?
Some censors (e.g. Roskomnadzor) monitor connections from domestic IPs to foreign
servers more aggressively than domestic-to-domestic traffic. A common workaround is
to split the proxy across two servers:
- **Front server** — domestic (or neutral) IP, accepts Telegram client connections.
- **Back server** — foreign IP, connects to Telegram data centres.
```
Telegram client
│
▼ (443 / any port)
┌────────────┐
│ front node │ domestic server (mtp_handler, session storage, policies)
└─────┬──────┘
│ inter-server link (VPN or TLS)
▼
┌────────────┐
│ back node │ foreign server (DC pool connections to Telegram)
└─────┬──────┘
│ TCP to Telegram
▼
Telegram DC
```
Multiple front servers can share one back server — just set the same `back_node`
address in each front's config. The DC pools on the back multiplex all client
connections regardless of which front they came from.
### Prerequisites
- Erlang/OTP 25+ installed on **both** servers (same version recommended).
- Both servers can reach each other over TCP (the inter-server port, see below).
- The back server has outbound TCP access to Telegram's infrastructure (ports
are announced dynamically in [Telegram's proxy config](https://core.telegram.org/getProxyConfig).
### Step 1 — secure the inter-server link
The two servers communicate using Erlang's built-in distribution protocol, which
allows full remote control of the process. **You must restrict access to this
channel** to the two proxy servers only. There are two ways to do this:
#### Option A: Censorship-resistant tunnel (recommended if front is in Russia)
Standard VPN protocols (WireGuard, OpenVPN) are detectable and blocked on many
Russian ISPs. Use a DPI-resistant tunnel instead:
- **[Shadowsocks](https://shadowsocks.org/)** — widely used, low overhead
- **[VLESS/XRay](https://github.com/XTLS/Xray-core)** — highly configurable, very hard to block
- **[Hysteria2](https://github.com/apernet/hysteria)** — QUIC-based, good for lossy links
Set up the tunnel between front and back servers and use the **tunnel interface
addresses** in the node names (`front@10.8.0.1`, `back@10.8.0.2`). No extra
Erlang config is needed once the tunnel is up.
> If the front server is **not** in a heavily censored region, WireGuard or
> IPsec work equally well and are simpler to set up.
#### Option B: TLS distribution (no tunnel required)
If you prefer not to run a separate tunnel, you can secure the distribution link
with mutual-TLS certificates. Run on the **back server**:
```bash
# Step 1 — on the back server: initialise CA and generate back cert
./scripts/gen_dist_certs.sh init /etc/mtproto-proxy/dist
# Step 2 — repeat for each front server you add
./scripts/gen_dist_certs.sh add-node /etc/mtproto-proxy/dist front
# (use a distinct name per front, e.g. front1, front2, …)
```
Copy to each server (paths already substituted — no editing needed):
* back server: ca.pem back.pem back.key ssl_dist.back.conf
* front server: ca.pem front.pem front.key ssl_dist.front.conf
Place all files in `/etc/mtproto-proxy/dist/`.
Then uncomment `-proto_dist` and `-ssl_dist_optfile` in `vm.args` on each server.
Reference: [Erlang TLS distribution docs](https://www.erlang.org/doc/apps/ssl/ssl_distribution.html)
### Step 2 — configure the back server
Run on the **back server**:
```bash
make init-config ROLE=back
```
This copies `config/sys.config.back.example` → `config/prod-sys.config` and
`config/vm.args.back.example` → `config/prod-vm.args`. Edit them:
- In `vm.args`: set the **back** server's IP: `-name back@<BACK_IP>` and choose
a strong cookie string (`-setcookie ...`).
- In `sys.config`: set `external_ip` to the back server's public IP, or leave
`ip_lookup_services` to auto-detect it.
- If using TLS distribution (Option B): uncomment the `-proto_dist` /
`-ssl_dist_optfile` lines in `vm.args`.
### Step 3 — configure the front server
Run on the **front server**:
```bash
make init-config ROLE=front
```
This copies `config/sys.config.front.example` → `config/prod-sys.config` and
`config/vm.args.front.example` → `config/prod-vm.args`. Edit them:
- In `vm.args`: set the **front** server's IP: `-name front@<FRONT_IP>` and the
**same** cookie string as the back.
- In `sys.config`: set `back_node` to the back node name you chose above
(e.g. `'back@10.8.0.2'`), configure `ports` / `secret` / `tag` as usual.
- If using TLS distribution: uncomment `-proto_dist` / `-ssl_dist_optfile` here too.
### Step 4 — start in order
Always **start the back server first**. The front server connects to it on
startup; if the back is not yet up the front will log a warning and retry
automatically on the next client connection.
```bash
# On back server:
make ROLE=back && sudo make install && systemctl start mtproto-proxy
# On front server (after back is up):
make ROLE=front && sudo make install && systemctl start mtproto-proxy
```
> **Why `ROLE=` on every `make`?** After `git pull`, if a new release updates
> `config/sys.config.back.example`, `make ROLE=back` detects the change (target
> is older than its prerequisite), shows a diff, and prompts before overwriting
> your `prod-sys.config`. Plain `make` (default `ROLE=both`) compares against
> `sys.config.example` instead and silently misses changes to the back/front
> templates.
### Step 5 — verify
On the front server, check that the back node is visible:
```bash
/opt/personal_mtproxy/bin/mtproto_proxy remote_console
# In the Erlang shell:
nodes(). % should list the back node
```
On the back server, verify DC pools are running:
```bash
/opt/personal_mtproxy/bin/mtproto_proxy remote_console
# In the Erlang shell:
mtp_config:status().
```
### Firewall rules
| Server | Port | Allow from |
|--------|-----------------------|---------------|
| back | 4369 (EPMD) | front IP only |
| back | 9199 (dist) | front IP only |
| front | 4369 (EPMD) | back IP only |
| front | 9199 (dist) | back IP only |
| front | 443 / your proxy port | anywhere |
If you used a fixed dist port in `vm.args` (`inet_dist_listen_min/max 9199`),
only port 9199 needs to be open; otherwise allow the full 9199–9254 range plus
EPMD (4369). **Never expose the distribution port to the public internet.**

11
config/vm.args.back.example
View file

@ -30,15 +30,16 @@
## Option B: TLS distribution (no tunnel required)
## - Generate certificates: scripts/gen_dist_certs.sh init /etc/mtproto-proxy/dist
## - Place ca.pem, back.pem, back.key in /etc/mtproto-proxy/dist/
## - Deploy ssl_dist.back.conf as /etc/mtproto-proxy/dist/ssl_dist.conf
## - Place ssl_dist.back.conf in /etc/mtproto-proxy/dist/ on the back server.
## - On each front node run: scripts/gen_dist_certs.sh add-node /etc/mtproto-proxy/dist front
## - Uncomment the lines below:
##
## -proto_dist inet_tls
## -ssl_dist_optfile /etc/mtproto-proxy/dist/ssl_dist.conf
# -proto_dist inet_tls
# -ssl_dist_optfile /etc/mtproto-proxy/dist/ssl_dist.back.conf
##
## Firewall: allow TCP on the distribution port (9199 below) only between
## the front and back servers, never to the public internet.
##
## -kernel inet_dist_listen_min 9199
## -kernel inet_dist_listen_max 9199
# -kernel inet_dist_listen_min 9199
# -kernel inet_dist_listen_max 9199
## -----------------------------------------------------------------------

15
config/vm.args.front.example
View file

@ -28,17 +28,18 @@
## - If front is NOT in a censored region, WireGuard/IPsec work fine too.
##
## Option B: TLS distribution (no tunnel required)
## - On back server: scripts/gen_dist_certs.sh init /etc/mtproto-proxy/dist
## - Run per front: scripts/gen_dist_certs.sh add-node /etc/mtproto-proxy/dist front
## - On back server: `scripts/gen_dist_certs.sh init /etc/mtproto-proxy/dist`
## - Run per front: `scripts/gen_dist_certs.sh add-node /etc/mtproto-proxy/dist front`
## - Place ca.pem, front.pem, front.key in /etc/mtproto-proxy/dist/ here.
## - Deploy ssl_dist.front.conf as /etc/mtproto-proxy/dist/ssl_dist.conf
## - Place ssl_dist.front.conf in /etc/mtproto-proxy/dist/ on the front server.
## - Uncomment the lines below:
##
## -proto_dist inet_tls
## -ssl_dist_optfile /etc/mtproto-proxy/dist/ssl_dist.conf
# -proto_dist inet_tls
# -ssl_dist_optfile /etc/mtproto-proxy/dist/ssl_dist.front.conf
##
## Firewall: allow TCP on the distribution port (9199 below) only between
## the front and back servers, never to the public internet.
##
## -kernel inet_dist_listen_min 9199
## -kernel inet_dist_listen_max 9199
# -kernel inet_dist_listen_min 9199
# -kernel inet_dist_listen_max 9199
## -----------------------------------------------------------------------

19
doc/handler-downstream-flow.md
View file

@ -9,19 +9,30 @@ and the steady-state data flow that follows.
- `mtp_down_conn` — multiplexed TCP connection to a Telegram DC
- `Telegram DC` — the upstream Telegram data-centre server
In **split mode** (`node_role = front / back`) `mtp_handler` runs on the front
node and `mtp_dc_pool` / `mtp_down_conn` run on the back node. The pool is
addressed as `{mtp_dc_pool_N, BackNode}` — Erlang distribution makes the
`gen_server:call` and all subsequent casts transparent across nodes. Multiple
front nodes can share the same back node; the pools multiplex over all
upstream connections regardless of origin.
```mermaid
sequenceDiagram
participant Client as Telegram client
participant Handler as mtp_handler
participant Pool as mtp_dc_pool
participant Down as mtp_down_conn
box LightBlue "FRONT node"
participant Handler as mtp_handler
end
box LightGreen "BACK node"
participant Pool as mtp_dc_pool
participant Down as mtp_down_conn
end
participant TG as Telegram DC
Client->>Handler: TCP connect + Hello bytes
Note over Handler: decode protocol headers<br/>(fake-TLS / obfuscated / secure)<br/>stage: hello → tunnel
Note over Handler: resolve pool: whereis(dc_to_pool_name(DcId))<br/>(registered name lookup; falls back to default DC from mtp_config if not found)
Note over Handler: resolve pool:<br/>single-node: whereis(dc_to_pool_name(DcId))<br/>split mode: erpc:call(BackNode, erlang, whereis, [PoolName])<br/>→ returns {PoolName, BackNode}<br/>(falls back to default DC from mtp_config if not found)
Handler->>Pool: mtp_dc_pool:get(Pool, self(), Opts) [sync]
Pool-->>Down: upstream_new(Handler, Opts) [cast]
Pool->>Handler: Downstream pid

21
doc/migration-flow.md
View file

@ -11,13 +11,26 @@ surviving (or freshly-started) DC connection transparently.
- `mtp_handler` — one process per connected Telegram client
- `mtp_down_conn (new)` — replacement downstream spawned by the pool
**Split-mode note:** in `front/back` split mode `mtp_handler` lives on the
front node and `mtp_dc_pool` / `mtp_down_conn` live on the back node. Every
message in the diagram below that crosses the front↔back boundary (the
`migrate` cast, `upstream_new` cast, `Pool->>Handler` reply, etc.) is carried
transparently by Erlang distribution — no code changes are needed because
Erlang PIDs and `gen_server` calls work across nodes unchanged. Process
monitors also fire on node disconnection, so a back-node restart causes all
affected front-node handlers to exit cleanly.
```mermaid
sequenceDiagram
participant TG as Telegram
participant OldDown as mtp_down_conn (old)
participant Pool as mtp_dc_pool
participant Handler as mtp_handler
participant NewDown as mtp_down_conn (new)
box LightGreen "BACK node"
participant OldDown as mtp_down_conn (old)
participant Pool as mtp_dc_pool
participant NewDown as mtp_down_conn (new)
end
box LightBlue "FRONT node"
participant Handler as mtp_handler
end
TG->>OldDown: TCP close

144
scripts/gen_dist_certs.sh Executable file
View file

@ -0,0 +1,144 @@
#!/usr/bin/env bash
# gen_dist_certs.sh — generate TLS certificates for Erlang distribution
#
# This script is designed to be run in TWO steps:
#
# Step 1 — run on the BACK server (once, to set up the CA and back cert):
# ./scripts/gen_dist_certs.sh init <output_dir>
#
# Step 2 — run on the BACK server for each FRONT server you add:
# ./scripts/gen_dist_certs.sh add-node <output_dir> <node_name>
#
# Example — back server and two front servers:
# ./scripts/gen_dist_certs.sh init /etc/mtproto-proxy/dist
# ./scripts/gen_dist_certs.sh add-node /etc/mtproto-proxy/dist front1
# ./scripts/gen_dist_certs.sh add-node /etc/mtproto-proxy/dist front2
#
# Output files (all in <output_dir>):
# ca.pem — CA certificate (copy to every server)
# back.pem / back.key — back node cert/key (keep on back server)
# <name>.pem / <name>.key — per-front cert/key (copy to that front server)
# ssl_dist.back.conf — ready-to-use Erlang TLS config for back
# (copy to /etc/mtproto-proxy/dist/ on back server)
# ssl_dist.<name>.conf — ready-to-use Erlang TLS config for that front
# (copy to /etc/mtproto-proxy/dist/ on that front server)
#
# After setup:
# On back server: ca.pem back.pem back.key ssl_dist.back.conf
# On each front: ca.pem <name>.pem <name>.key ssl_dist.<name>.conf
# Uncomment -proto_dist and -ssl_dist_optfile in vm.args on each server.
#
# Reference: https://www.erlang.org/doc/apps/ssl/ssl_distribution.html
set -euo pipefail
usage() {
echo "Usage:" >&2
echo " $0 init <output_dir> # Step 1: CA + back cert" >&2
echo " $0 add-node <output_dir> <node_name> # Step 2: add a front cert" >&2
exit 1
}
[ $# -lt 2 ] && usage
CMD="$1"
OUTDIR="$2"
DAYS=3650 # 10 years
mkdir -p "$OUTDIR"
# ---- shared: ensure CA exists ----
ensure_ca() {
if [ ! -f "$OUTDIR/ca.key" ]; then
echo "==> Generating CA key and certificate..."
openssl req -new -x509 -days "$DAYS" \
-keyout "$OUTDIR/ca.key" \
-out "$OUTDIR/ca.pem" \
-nodes \
-subj "/CN=mtproto-proxy-dist-ca"
chmod 600 "$OUTDIR/ca.key"
else
echo "==> Using existing CA: $OUTDIR/ca.key"
fi
}
# ---- shared: generate one node cert signed by the CA ----
gen_node_cert() {
local NAME="$1"
echo "==> Generating certificate for node: $NAME"
openssl req -new \
-keyout "$OUTDIR/$NAME.key" \
-out "$OUTDIR/$NAME.csr" \
-nodes \
-subj "/CN=$NAME"
openssl x509 -req -days "$DAYS" \
-in "$OUTDIR/$NAME.csr" \
-CA "$OUTDIR/ca.pem" \
-CAkey "$OUTDIR/ca.key" \
-CAcreateserial \
-out "$OUTDIR/$NAME.pem"
rm -f "$OUTDIR/$NAME.csr"
chmod 600 "$OUTDIR/$NAME.key"
echo " -> $OUTDIR/$NAME.pem $OUTDIR/$NAME.key"
}
# ---- shared: write a ready-to-use ssl_dist.<name>.conf for one node ----
write_ssl_conf() {
local NAME="$1"
local CONF="$OUTDIR/ssl_dist.$NAME.conf"
cat > "$CONF" << EOF
%% Erlang TLS distribution config for node: $NAME
%% Copy this file to /etc/mtproto-proxy/dist/ on the '$NAME' server.
[
{server,
[{certfile, "/etc/mtproto-proxy/dist/$NAME.pem"},
{keyfile, "/etc/mtproto-proxy/dist/$NAME.key"},
{cacertfile, "/etc/mtproto-proxy/dist/ca.pem"},
{verify, verify_peer},
{fail_if_no_peer_cert, true}]},
{client,
[{certfile, "/etc/mtproto-proxy/dist/$NAME.pem"},
{keyfile, "/etc/mtproto-proxy/dist/$NAME.key"},
{cacertfile, "/etc/mtproto-proxy/dist/ca.pem"},
{verify, verify_peer}]}
].
EOF
echo " -> $CONF"
}
case "$CMD" in
init)
ensure_ca
gen_node_cert "back"
write_ssl_conf "back"
echo ""
echo "==> Done. Back server setup complete."
echo ""
echo "Next:"
echo " 1. On the back server, copy to /etc/mtproto-proxy/dist/:"
echo " ca.pem back.pem back.key"
echo " 2. Copy ssl_dist.back.conf to /etc/mtproto-proxy/dist/"
echo " 3. Uncomment -proto_dist / -ssl_dist_optfile in vm.args."
echo " 4. For each front server, run:"
echo " $0 add-node $OUTDIR <front_name>"
;;
add-node)
[ $# -lt 3 ] && usage
NAME="$3"
if [ ! -f "$OUTDIR/ca.key" ]; then
echo "ERROR: CA not found in $OUTDIR. Run '$0 init $OUTDIR' first." >&2
exit 1
fi
gen_node_cert "$NAME"
write_ssl_conf "$NAME"
echo ""
echo "==> Done. Certificate for '$NAME' generated."
echo ""
echo "Copy to the '$NAME' front server:"
echo " ca.pem $NAME.pem $NAME.key"
echo " ssl_dist.$NAME.conf (copy to /etc/mtproto-proxy/dist/ on that server)"
;;
*)
usage
;;
esac