Commit graph

224 commits

Author SHA1 Message Date
Sergey Prokhorov
154a8fb8e6
Track downstream passive backpressure
* Add histogram metrics tracking passive state duration
* Add config option to tweak "upstream" TCP send timeout

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-14 15:08:31 +02:00
Sergey Prokhorov
b3b2298117
Add mtp_ping escript and modernise FakeTLS ClientHello
- Add src/mtp_ping.erl: standalone escript that pings a Telegram MTProto
  proxy across configurable DC IDs and protocols (normal/secure/fake-tls).
  Accepts all proxy URL formats (tg://proxy and https://t.me/proxy, hex and
  base64 secrets). Prints per-attempt timings (TCP/Handshake/Ping/Total) and
  a two-section summary (protocol status + per-DC averages). Supports
  --dc, --proto, --timeout, --repeat, --verbose flags.

- Modernise mtp_fake_tls:make_client_hello/4 to match tdesktop commit
  b72deb1 + tdlib commit d0de8a7:
  - ML-KEM-768 key share (X25519MLKEM768 group 0x11ec, 1184-byte key)
  - Updated supported_groups: GREASE + X25519MLKEM768 + x25519 + secp256r1 + secp384r1
  - Full extension set (17 extensions including ALPS, SCT, status_request, …)
  - ECH outer extension type 0xfe0d (was 0xfe02), random field 32 bytes (was 20)
  - ALPS type 0x44cd (was 0x4469)
  - Variable-length output (~1776 bytes), no fixed padding
  - Extension order shuffled per-connection (match tdesktop fingerprint evasion)
  - supported_versions: TLS 1.3 + 1.2 only (drop 1.0 and 1.1)
  - Add compress_certificate extension (brotli)

- Remove legacy make_client_hello/3 and /5 (fixed-padding format) and
  add_padding_ext/2; update prop_mtp_fake_tls and mtp_test_client/
  single_dc_SUITE to use the modern /2 and /4 arities.

- Remove ifdef(TEST) guards from mtp_obfuscated and mtp_fake_tls exports
  needed by mtp_ping; widen DC ID guard in mtp_obfuscated:client_create/4
  to full 16-bit signed range.

- Improve parse_server_hello to correctly handle fragmented TLS responses:
  replace the fixed 517-byte incomplete threshold with structure-aware
  record counting (tls_records_complete/2); distinguish tls_domain_forwarding
  (proxy forwarded to SNI host), tls_alert (proxy rejected ClientHello),
  and not_proxy_response; propagate these to mtp_ping error messages.

- Document mtp_ping in README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-14 04:11:56 +02:00
Sergey Prokhorov
cb23a30fe2
Warn if 'back' node is not accessible from 'front' 2026-04-14 02:26:20 +02:00
Sergey Prokhorov
e343e30060
Fix .app.src syntax (missing comma) 2026-04-12 04:04:20 +02:00
Sergey Prokhorov
0f042b27ad
Merge pull request #118 from centuriononon/feat/configurable-api-timeouts
Make core API HTTP timeout configurable
2026-04-12 03:52:49 +02:00
Sergey Prokhorov
0180d7e2af
Bump version 0.8.4 2026-04-12 01:04:41 +02:00
Sergey Prokhorov
d5f5a74de5
fix: start epmd before net_kernel in split_dc_SUITE; ci: epmd -daemon
split_dc_SUITE requires Erlang distribution (peer module). On GitHub
Actions (and any fresh environment) epmd is not pre-started, causing
net_kernel:start to fail with nodistribution.

Fix: call os:cmd("epmd -daemon") before net_kernel:start in
init_per_suite. The call is idempotent — safe when epmd is already
running.

Also start epmd explicitly in the CI 'ct' step as a belt-and-suspenders
measure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-12 00:39:02 +02:00
Sergey Prokhorov
121d8b7413
docs: split-mode setup guide, architecture diagrams, cert script, build
README:
- New 'Split-mode setup' section: motivation, firewall rules, step-by-step
  instructions for both VPN tunnel and TLS distribution options
- Split-mode bullet added to Features list
- Notes on DPI-resistant tunnels (Shadowsocks, VLESS/XRay, Hysteria2) for
  Russian deployment; standard VPN protocols (WireGuard, OpenVPN) may be blocked
- Install instructions updated to use `make init-config` (copies templates,
  auto-detects public IP) instead of manual cp; ROLE= documented throughout
- Split-mode Step 4 uses `make ROLE=back/front` so template-change detection
  works correctly after `git pull`

Makefile:
- ROLE ?= both variable selects config templates (both/front/back)
- Config prereq rules use $(SYS_CONFIG_SRC) / $(VM_ARGS_SRC) based on ROLE
- New `init-config` target: force-copies templates, auto-detects public IP,
  prints edit reminder; replaces manual cp in install workflow

scripts/gen_dist_certs.sh:
- Two-step workflow: `init <dir>` on back server (CA + back cert),
  `add-node <dir> <name>` per front server (cert signed by existing CA)
- Generates per-node ssl_dist.<name>.conf with paths substituted (no
  NODE_NAME placeholder to edit manually)
- ssl_dist.<name>.conf is now used directly (no rename to ssl_dist.conf);
  vm.args examples and README updated to match

config/vm.args.{front,back}.example:
- -ssl_dist_optfile points to role-specific filename (ssl_dist.front.conf /
  ssl_dist.back.conf) so cert files can be copied as-is without renaming

AGENTS.md:
- Role-overview Mermaid flowchart showing front/back/both process split
- Data-plane section replaced with links to doc/ (no duplication)
- Supervision tree, key interactions, split-mode config keys updated

doc/handler-downstream-flow.md, doc/migration-flow.md:
- Mermaid box grouping to visually separate FRONT and BACK node participants
- erpc:call reference corrected (was rpc:call)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-12 00:34:45 +02:00
Sergey Prokhorov
42d86517a4
Add split-mode: front/back nodes via Erlang distribution
Adds `node_role = front | back | both` (default `both` — no behaviour change).
In split mode the domestic front node runs Ranch listeners and session/policy
processes; the foreign back node runs mtp_config, DC pools, and downstream
connections.

The key implementation insight: Erlang's transparent distributed process
addressing means `gen_server:call({mtp_dc_pool_1, BackNode}, ...)` works
identically to a local call — zero changes to the hot-path message protocol.
Process monitors fire on node disconnection, so back-node restarts propagate
cleanly to front-node handlers without any watcher process.

Changes:
- mtproto_proxy_sup: role-parameterised children/1
- mtproto_proxy_app: role-gated start, config_changed, port management
- mtp_config: backend_node/0, remote-aware get_downstream_pool/1 and
  get_default_dc/0 using erpc:call; get_downstream_safe/2 dispatches
  remotely in front mode
- mtp_metric: passive_metrics/0 skips unavailable gauges per role
- split_dc_SUITE: two CT cases (echo, migration) on OTP peer nodes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 23:33:09 +02:00
Sergey Prokhorov
4e0db66fc1
mtp_config: parse default DC id from Telegram config; drop get_netloc_safe; ETS protected
- parse_config/1 now returns {DefaultDc, Downstreams} using a single
  lists:foldl pass; the 'default X;' line sets the fallback DC id
- update_ids/3 stores {default_dc, DcId} in ETS alongside dc_ids
- get_default_dc/0 reads default_dc from ETS (safe from any process)
- get_downstream_safe/2 fallback uses get_default_dc() instead of
  random_choice; errors immediately if default == requested (avoid loop)
- get_netloc_safe/1 removed: dead code since 2018, never called
- ETS table changed from public to protected (only mtp_config writes)
- doc/handler-downstream-flow.md: new sequence diagram + update note
  about pool resolution fallback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 18:35:02 +02:00
Sergey Prokhorov
8ce8f2196d
test: fix downstream_migration metric labels (add dc_id)
Metric was emitted with 3 labels [listener, dc_id, result] since
3c29fa3 (Include dc_id to migration metric) but tests were still
matching only [listener, result], causing wait_for_value to time out.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 18:35:02 +02:00
Sergey Prokhorov
3c29fa31e5
Include dc_id to migration metric
This is to see which DCs trigger migrations more often.
In UI should be divided by number of clients of DC
2026-04-10 00:50:00 +02:00
Sergey Prokhorov
16abd21927
metrics: declare downstream_migration counter
Metric was emitted in 492956a (feat: transparent client migration on DC
connection death) but never declared in active_metrics/0. Labels:
[listener, result] where result is ok | empty | not_found | mid_send.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-09 02:09:10 +02:00
Sergey Prokhorov
f2bdf3cb7f
Fix metrics call to ranch:info()
Bug was introduced when we upgraded to ranch 2.x.x in
fc9ec1d326
2026-04-09 00:48:02 +02:00
Sergey Prokhorov
dfa991f803
docs: add cross-references to migration-flow.md and debug logging tip
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-08 01:49:35 +02:00
Sergey Prokhorov
492956a598
feat: transparent client migration on DC connection death
When Telegram closes a DC TCP connection, instead of dropping all
multiplexed clients, the proxy now remaps them to a surviving (or
freshly-spawned) replacement DC connection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-08 00:46:56 +02:00
Sergey Prokhorov
ed2fa40ee5
Add TCP keepalive tuning and 4-element UpsStatic
- TCP keepalive on downstream connections (KEEPIDLE=40s, KEEPINTVL=40s,
  KEEPCNT=5): matches reference C implementation (net-events.c).
  Uses raw socket options for OTP 25+ compatibility (named keepidle/
  keepintvl/keepcnt require OTP 28.3+/Kernel 10.5).

- Expand UpsStatic to 4-element tuple {ConnId, Addr, AdTag, PacketLayer}
  and thread packet_layer through mtp_handler -> mtp_dc_pool -> mtp_down_conn.
  Fixes pattern match in handle_upstream_closed.

- Reduce log noise for expected DC connection rotation:
  When Telegram closes a downstream connection with no active clients,
  stop with {shutdown, downstream_socket_closed} instead of bare atom;
  mtp_down_conn logs INFO, mtp_dc_pool logs INFO (was ERROR for all).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 19:12:59 +02:00
Sergey Prokhorov
84cceaee48
Fix RPC protocol flags and add RPC_PING/PONG response
- protocol_flag/1: map client protocol to correct RPC_PROXY_REQ flags
  mtp_abridged     -> FLAG_ABRIDGED (0x40000000 = RPC_F_COMPACT)
  mtp_intermediate -> FLAG_INTERMEDIATE (0x20000000 = RPC_F_MEDIUM)
  mtp_secure       -> FLAG_INTERMEDIATE | FLAG_PAD (0x28000000)
  Previously always sent FLAG_ABRIDGED regardless of protocol.

- Decode RPC_PING from Telegram and respond with RPC_PONG
  Matches reference C implementation (tcp_rpcs_default_execute).

- Expand UpsStatic tuple to 4 elements (ConnId, Addr, AdTag, Protocol)
  so protocol is available at encode time.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 19:12:28 +02:00
Sergey Prokhorov
2dd6e6d589
Fix deprecated code:lib_dir/2 warning in mtp_test_datacenter
Replace code:lib_dir(mtproto_proxy, test) with
filename:join(code:lib_dir(mtproto_proxy), "test") as
the two-argument form is deprecated since OTP 27.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 13:48:07 +02:00
Sergey Prokhorov
dfe8ebf034
Send TLS decode_error alert on malformed ClientHello and missing SNI
Scanners probe for fake-TLS proxies by sending structurally malformed
ClientHellos (e.g. ExtensionsLen=0 with trailing extension bytes). A
real TLS server responds with a fatal decode_error alert; previously
the proxy crashed the handler process silently, making it detectable.

Changes:
- mtp_fake_tls: add TLS_REC_ALERT, TLS_ALERT_FATAL, TLS_ALERT_DECODE_ERROR
  macros; export tls_decode_error_alert/0 which builds the 7-byte alert
  frame from macros
- mtp_fake_tls: add second clause to parse_client_hello/1 that throws
  {protocol_error, tls_bad_client_hello, bad_client_hello} instead of
  letting a bare function_clause propagate
- mtp_fake_tls: tighten parse_sni/1 catch to match the specific tagged
  error rather than a catch-all error:_
- mtp_handler: add attempt_fronting clauses for tls_bad_client_hello and
  tls_no_sni — both send the decode_error alert before closing
- mtp_handler: effective_secret/2 now raises tls_bad_client_hello (not
  tls_invalid_digest) when per_sni_secrets=on and the ClientHello has
  no SNI, so it also gets the alert treatment
- single_dc_SUITE: new malformed_tls_hello_decode_error_case/1 verifies
  the alert bytes are sent and the metric is incremented
- AGENTS.md: document test organisation, process architecture diagram,
  and upstream/downstream naming note

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 13:46:39 +02:00
Sergey Prokhorov
36b30e3f5f
Add per-SNI derived secrets feature
Each fake-TLS SNI domain gets a unique 16-byte secret derived from the
SNI, base secret and a private salt, so users cannot extract the base secret
from their proxy link or forge tokens by guessing other domains.

Derivation: SHA256(salt || hex(base_secret) || sni_domain)[0:16]
Salt-first ordering ensures security even when base_secret is known
(e.g. when non-fake-TLS protocols are enabled on the same port).

New config options (default: off):
  {per_sni_secrets, off | on}
  {per_sni_secret_salt, <<"..ascii..">>}

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 03:22:57 +02:00
Sergey Prokhorov
90cc0423ad
docker: upgrade to erlang:27-alpine and alpine:3.22
- Builder: erlang:21-alpine -> erlang:27-alpine (OTP 25+ required)
- Runtime: alpine:3.9 -> alpine:3.22 (matches builder's Alpine for ERTS ABI compatibility)
- Use ./rebar3 (project-bundled) instead of system rebar3
- Consolidate apk add calls in runtime stage
- Add libstdc++ to runtime: OTP 24+ JIT compiler (beam.smp) links against
  libstdc++ and libgcc_s, which are not present in the base Alpine image

Tested: built and pushed seriyps/mtproto-proxy:0.8.3 and :latest to Docker Hub.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 01:00:34 +02:00
Sergey Prokhorov
d0a81b04a5
Bump version to 0.8.3 2026-04-06 17:57:07 +02:00
Sergey Prokhorov
26fe4bc0e8
bugfix: Clear .hello_acc in mtp_handler when no longer needed
Before fix accumulator was never cleaned, taking memory.
Clear the accumulator on transition to tunnel stage (handshake complete)
and on transition to fronting stage (data already forwarded to front_sock).
2026-04-06 17:43:28 +02:00
Sergey Prokhorov
f24b70db80
Bump version to 0.8.2; add compatibility Ranch 1.8/2.0 2026-04-05 13:54:59 +02:00
Sergey Prokhorov
fc9ec1d326
Upgrade ranch 1.7.0 → 2.2.0
Ranch 2.x breaking changes addressed:
- Protocol callback changed from start_link/4 (Ref, Socket, Transport, Opts)
  to start_link/3 (Ref, Transport, Opts); socket obtained via ranch:handshake/1
- ranch:info/0 now returns #{Name => #{...}} instead of [{Name, [proplists]}];
  updated mtp_listeners/0, running_ports/0, and config_change_case test

Also update AGENTS.md with CT failure debugging workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-05 01:41:01 +02:00
Sergey Prokhorov
683732e584
Bump version to 0.8.1. Remove remnants of lager 2026-04-04 00:52:44 +02:00
Sergey Prokhorov
81aba02ad4
Add warning about infinite loop in sni domain fronting 2026-04-04 00:49:16 +02:00
Sergey Prokhorov
b39f78134a
Use rebar3 3.24.0 for compatibility with OTP-25 2026-04-03 20:12:44 +02:00
Sergey Prokhorov
9cf3e9e847
Add domain fronting for fake-TLS connections
When a fake-TLS handshake fails (wrong secret, DPI probe, replay attack),
forward the raw TCP connection transparently to the SNI host instead of
closing — making the proxy indistinguishable from a normal HTTPS server.
Replay detection is moved to ClientHello level (before ServerHello) to
allow clean forwarding. Controlled by {domain_fronting, off|sni|"host:port"}.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-03 19:35:31 +02:00
centurion
60866a6da2 feat: impl new core_api_http_timeout_ms option 2026-03-20 19:21:49 -03:00
Sergey Prokhorov
421a6cc90c
Merge pull request #115 from centuriononon/fix/dc-ids-range
Fix/dc ids range
2026-02-26 12:33:58 +01:00
centurion
e25c0a3bc7 Deleted .run dir 2026-02-26 02:33:31 -03:00
centurion
f98eaedfd4 Gitignore .run directory 2026-02-26 02:32:44 -03:00
centurion
313873bc3f Fixed DC IDs range
Telegram Datacenter IDs currently include values out of -9..9 which leads to crash on launch
2026-02-26 02:28:40 -03:00
Sergey Prokhorov
9a286862c8
Use OTP logger instead of lager. 2026-02-17 01:50:54 +01:00
Sergey Prokhorov
0cc2e02c8c
Modernisations
* Add variable-length ClientHello
* Make it compile on OTP 27+
* Upgrade OTP versions in CI
2026-02-17 01:11:32 +01:00
Sergey Prokhorov
f9c2d32d4f
Test on OTP-26, drop pre-OTP-23; OTP-27 not compatible yet; fix dialyzer 2024-12-04 14:53:55 +01:00
Sergey Prokhorov
9072be78bd
Use updated upload-artifact GH action 2024-12-04 12:31:57 +01:00
Sergey Prokhorov
247bd9339b
Replace seriyps.ru with seriyps.com
The old domain no longer works
2024-12-04 12:28:56 +01:00
Sergey Prokhorov
37497fff04
Accept unexpected RPC packets from Telegram backend 2022-11-23 01:44:16 +01:00
Sergey Prokhorov
9cb1984889
Merge pull request #77 from kianmeng/fix-typos
Fix typos
2022-11-23 01:03:52 +01:00
Sergey Prokhorov
a2e62fc12c
Merge pull request #85 from seriyps/github-ci
Replace Travis-CI with Github actions
2022-11-23 01:03:04 +01:00
Sergey Prokhorov
5fcedfdf9c
Replace Travis-CI with Github actions, add support for OTP-25 2022-11-23 00:58:53 +01:00
Kian-Meng Ang
bab411303a Fix typos 2022-04-05 21:16:13 +08:00
Sergey Prokhorov
a07e03f73a
Merge pull request #68 from T1me/master
fix README
2021-09-26 18:23:40 +02:00
T1me
a3a81ee846
fix README 2021-09-26 21:25:43 +08:00
Sergey Prokhorov
8104202a7b
Create FUNDING.yml 2021-09-07 19:00:57 +02:00
Sergey Prokhorov
2f2f2bb6b0
Migrate to more modern OTP versions (remove OTP-20, add OTP-23,24) 2021-05-19 20:50:21 +02:00
Sergey Prokhorov
5ad7c536da
Add option to disable crc32 check in mtp_full to save CPU 2019-10-24 23:28:24 +02:00