Commit graph

55 commits

Author SHA1 Message Date
Sergey Prokhorov
154a8fb8e6
Track downstream passive backpressure
* Add histogram metrics tracking passive state duration
* Add config option to tweak "upstream" TCP send timeout

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-14 15:08:31 +02:00
Sergey Prokhorov
b3b2298117
Add mtp_ping escript and modernise FakeTLS ClientHello
- Add src/mtp_ping.erl: standalone escript that pings a Telegram MTProto
  proxy across configurable DC IDs and protocols (normal/secure/fake-tls).
  Accepts all proxy URL formats (tg://proxy and https://t.me/proxy, hex and
  base64 secrets). Prints per-attempt timings (TCP/Handshake/Ping/Total) and
  a two-section summary (protocol status + per-DC averages). Supports
  --dc, --proto, --timeout, --repeat, --verbose flags.

- Modernise mtp_fake_tls:make_client_hello/4 to match tdesktop commit
  b72deb1 + tdlib commit d0de8a7:
  - ML-KEM-768 key share (X25519MLKEM768 group 0x11ec, 1184-byte key)
  - Updated supported_groups: GREASE + X25519MLKEM768 + x25519 + secp256r1 + secp384r1
  - Full extension set (17 extensions including ALPS, SCT, status_request, …)
  - ECH outer extension type 0xfe0d (was 0xfe02), random field 32 bytes (was 20)
  - ALPS type 0x44cd (was 0x4469)
  - Variable-length output (~1776 bytes), no fixed padding
  - Extension order shuffled per-connection (match tdesktop fingerprint evasion)
  - supported_versions: TLS 1.3 + 1.2 only (drop 1.0 and 1.1)
  - Add compress_certificate extension (brotli)

- Remove legacy make_client_hello/3 and /5 (fixed-padding format) and
  add_padding_ext/2; update prop_mtp_fake_tls and mtp_test_client/
  single_dc_SUITE to use the modern /2 and /4 arities.

- Remove ifdef(TEST) guards from mtp_obfuscated and mtp_fake_tls exports
  needed by mtp_ping; widen DC ID guard in mtp_obfuscated:client_create/4
  to full 16-bit signed range.

- Improve parse_server_hello to correctly handle fragmented TLS responses:
  replace the fixed 517-byte incomplete threshold with structure-aware
  record counting (tls_records_complete/2); distinguish tls_domain_forwarding
  (proxy forwarded to SNI host), tls_alert (proxy rejected ClientHello),
  and not_proxy_response; propagate these to mtp_ping error messages.

- Document mtp_ping in README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-14 04:11:56 +02:00
Sergey Prokhorov
d5f5a74de5
fix: start epmd before net_kernel in split_dc_SUITE; ci: epmd -daemon
split_dc_SUITE requires Erlang distribution (peer module). On GitHub
Actions (and any fresh environment) epmd is not pre-started, causing
net_kernel:start to fail with nodistribution.

Fix: call os:cmd("epmd -daemon") before net_kernel:start in
init_per_suite. The call is idempotent — safe when epmd is already
running.

Also start epmd explicitly in the CI 'ct' step as a belt-and-suspenders
measure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-12 00:39:02 +02:00
Sergey Prokhorov
42d86517a4
Add split-mode: front/back nodes via Erlang distribution
Adds `node_role = front | back | both` (default `both` — no behaviour change).
In split mode the domestic front node runs Ranch listeners and session/policy
processes; the foreign back node runs mtp_config, DC pools, and downstream
connections.

The key implementation insight: Erlang's transparent distributed process
addressing means `gen_server:call({mtp_dc_pool_1, BackNode}, ...)` works
identically to a local call — zero changes to the hot-path message protocol.
Process monitors fire on node disconnection, so back-node restarts propagate
cleanly to front-node handlers without any watcher process.

Changes:
- mtproto_proxy_sup: role-parameterised children/1
- mtproto_proxy_app: role-gated start, config_changed, port management
- mtp_config: backend_node/0, remote-aware get_downstream_pool/1 and
  get_default_dc/0 using erpc:call; get_downstream_safe/2 dispatches
  remotely in front mode
- mtp_metric: passive_metrics/0 skips unavailable gauges per role
- split_dc_SUITE: two CT cases (echo, migration) on OTP peer nodes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 23:33:09 +02:00
Sergey Prokhorov
8ce8f2196d
test: fix downstream_migration metric labels (add dc_id)
Metric was emitted with 3 labels [listener, dc_id, result] since
3c29fa3 (Include dc_id to migration metric) but tests were still
matching only [listener, result], causing wait_for_value to time out.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 18:35:02 +02:00
Sergey Prokhorov
492956a598
feat: transparent client migration on DC connection death
When Telegram closes a DC TCP connection, instead of dropping all
multiplexed clients, the proxy now remaps them to a surviving (or
freshly-spawned) replacement DC connection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-08 00:46:56 +02:00
Sergey Prokhorov
84cceaee48
Fix RPC protocol flags and add RPC_PING/PONG response
- protocol_flag/1: map client protocol to correct RPC_PROXY_REQ flags
  mtp_abridged     -> FLAG_ABRIDGED (0x40000000 = RPC_F_COMPACT)
  mtp_intermediate -> FLAG_INTERMEDIATE (0x20000000 = RPC_F_MEDIUM)
  mtp_secure       -> FLAG_INTERMEDIATE | FLAG_PAD (0x28000000)
  Previously always sent FLAG_ABRIDGED regardless of protocol.

- Decode RPC_PING from Telegram and respond with RPC_PONG
  Matches reference C implementation (tcp_rpcs_default_execute).

- Expand UpsStatic tuple to 4 elements (ConnId, Addr, AdTag, Protocol)
  so protocol is available at encode time.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 19:12:28 +02:00
Sergey Prokhorov
2dd6e6d589
Fix deprecated code:lib_dir/2 warning in mtp_test_datacenter
Replace code:lib_dir(mtproto_proxy, test) with
filename:join(code:lib_dir(mtproto_proxy), "test") as
the two-argument form is deprecated since OTP 27.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 13:48:07 +02:00
Sergey Prokhorov
dfe8ebf034
Send TLS decode_error alert on malformed ClientHello and missing SNI
Scanners probe for fake-TLS proxies by sending structurally malformed
ClientHellos (e.g. ExtensionsLen=0 with trailing extension bytes). A
real TLS server responds with a fatal decode_error alert; previously
the proxy crashed the handler process silently, making it detectable.

Changes:
- mtp_fake_tls: add TLS_REC_ALERT, TLS_ALERT_FATAL, TLS_ALERT_DECODE_ERROR
  macros; export tls_decode_error_alert/0 which builds the 7-byte alert
  frame from macros
- mtp_fake_tls: add second clause to parse_client_hello/1 that throws
  {protocol_error, tls_bad_client_hello, bad_client_hello} instead of
  letting a bare function_clause propagate
- mtp_fake_tls: tighten parse_sni/1 catch to match the specific tagged
  error rather than a catch-all error:_
- mtp_handler: add attempt_fronting clauses for tls_bad_client_hello and
  tls_no_sni — both send the decode_error alert before closing
- mtp_handler: effective_secret/2 now raises tls_bad_client_hello (not
  tls_invalid_digest) when per_sni_secrets=on and the ClientHello has
  no SNI, so it also gets the alert treatment
- single_dc_SUITE: new malformed_tls_hello_decode_error_case/1 verifies
  the alert bytes are sent and the metric is incremented
- AGENTS.md: document test organisation, process architecture diagram,
  and upstream/downstream naming note

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 13:46:39 +02:00
Sergey Prokhorov
36b30e3f5f
Add per-SNI derived secrets feature
Each fake-TLS SNI domain gets a unique 16-byte secret derived from the
SNI, base secret and a private salt, so users cannot extract the base secret
from their proxy link or forge tokens by guessing other domains.

Derivation: SHA256(salt || hex(base_secret) || sni_domain)[0:16]
Salt-first ordering ensures security even when base_secret is known
(e.g. when non-fake-TLS protocols are enabled on the same port).

New config options (default: off):
  {per_sni_secrets, off | on}
  {per_sni_secret_salt, <<"..ascii..">>}

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 03:22:57 +02:00
Sergey Prokhorov
fc9ec1d326
Upgrade ranch 1.7.0 → 2.2.0
Ranch 2.x breaking changes addressed:
- Protocol callback changed from start_link/4 (Ref, Socket, Transport, Opts)
  to start_link/3 (Ref, Transport, Opts); socket obtained via ranch:handshake/1
- ranch:info/0 now returns #{Name => #{...}} instead of [{Name, [proplists]}];
  updated mtp_listeners/0, running_ports/0, and config_change_case test

Also update AGENTS.md with CT failure debugging workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-05 01:41:01 +02:00
Sergey Prokhorov
9cf3e9e847
Add domain fronting for fake-TLS connections
When a fake-TLS handshake fails (wrong secret, DPI probe, replay attack),
forward the raw TCP connection transparently to the SNI host instead of
closing — making the proxy indistinguishable from a normal HTTPS server.
Replay detection is moved to ClientHello level (before ServerHello) to
allow clean forwarding. Controlled by {domain_fronting, off|sni|"host:port"}.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-03 19:35:31 +02:00
Sergey Prokhorov
9a286862c8
Use OTP logger instead of lager. 2026-02-17 01:50:54 +01:00
Sergey Prokhorov
0cc2e02c8c
Modernisations
* Add variable-length ClientHello
* Make it compile on OTP 27+
* Upgrade OTP versions in CI
2026-02-17 01:11:32 +01:00
Sergey Prokhorov
9cb1984889
Merge pull request #77 from kianmeng/fix-typos
Fix typos
2022-11-23 01:03:52 +01:00
Sergey Prokhorov
5fcedfdf9c
Replace Travis-CI with Github actions, add support for OTP-25 2022-11-23 00:58:53 +01:00
Kian-Meng Ang
bab411303a Fix typos 2022-04-05 21:16:13 +08:00
Sergey Prokhorov
5ad7c536da
Add option to disable crc32 check in mtp_full to save CPU 2019-10-24 23:28:24 +02:00
Sergey Prokhorov
7b4c5dde9c
tests: add encoder and decoder for unencrypted mtproto messages
req_pq and resPQ messages
2019-10-22 03:48:51 +02:00
Sergey Prokhorov
047193d806
Add benchmarks 2019-10-14 02:48:08 +02:00
Sergey Prokhorov
fb0a3bafc7
Fix bugs in policy, add tests 2019-08-20 23:41:44 +02:00
Sergey Prokhorov
30f7e6b9c1
Add fake-tls support to mtp_test_client; added TLS networking tests 2019-08-15 00:55:45 +02:00
Sergey Prokhorov
237f9f1d25
Limit TLS record size to 2^14
See rfc8446#section-5.1
2019-08-12 19:02:37 +02:00
Sergey Prokhorov
7677fe1150
Add 'listener' to protocol_error metric 2019-08-12 19:02:37 +02:00
Sergey Prokhorov
e28e21a77f
Fix decoding of packets split to multiple TLS frames. Fixes gh-16 2019-08-12 19:02:37 +02:00
Sergey Prokhorov
418709208c
Cosmetic changes: README, docker, dialyzer fixes 2019-08-12 19:02:37 +02:00
Sergey Prokhorov
0a1604d4be
Fake TLS protocol
* pretends being TLS1.3 + http2
* mtp_handler state-machine refactored
2019-08-12 19:00:28 +02:00
Sergey Prokhorov
c711054bf2
Add support for IPv6 clients. gh-11 2019-08-08 02:45:16 +02:00
Sergey Prokhorov
2371703207
Fix bug in mtp_codec
Buffer was not saved if crypto codec returns `incomplete`
2019-07-26 18:37:17 +02:00
Sergey Prokhorov
1429cfe267
Refactor codecs
* Get rid of per-codec internal read buffer
2019-07-25 13:17:37 +02:00
Сергей Прохоров
24ef827705
Add another replay attack protection: filter error replies from downstream 2019-05-26 03:47:15 +02:00
Сергей Прохоров
e66778ed7c
Make downstream backpressure tunable 2019-05-22 02:33:16 +02:00
Сергей Прохоров
e2308a6e57
Use "hut" for logger abstraction 2019-05-22 00:08:32 +02:00
Сергей Прохоров
0f4d180a06
Replay attack mitigation
Storage of recently used 1st packets added. Connections with the same
1st packet are disallowed
2019-05-20 03:14:32 +02:00
Сергей Прохоров
07d397ce93
tests: fix bug in common test reset code 2019-05-10 17:35:05 +02:00
Сергей Прохоров
977109d05a
Add special helpers for live config_change 2019-05-09 17:18:29 +02:00
Сергей Прохоров
f1070dfee2
Handle "max packet size" errors; fix related bug in abridged codec 2019-04-30 23:50:09 +02:00
Сергей Прохоров
90feb07cdf
Fix httpd "eaddrinuse" in tests 2019-04-02 00:02:10 +02:00
Сергей Прохоров
d8aec8155f
Add some statefull property-based integration tests 2019-03-13 02:12:21 +01:00
Сергей Прохоров
66f074655b
Separate metrics for different backpressure types; test tweaks 2019-03-08 00:44:35 +01:00
Сергей Прохоров
efafd37ca9
Tune socket receive buffer sizes to fix false positive backpressure 2019-03-08 00:27:21 +01:00
Сергей Прохоров
23ef74a551
Add downstream backpressure tests 2019-03-07 02:34:10 +01:00
Сергей Прохоров
e9471b25c2
tests: Make it possible to block until metric value will match predicate 2019-03-07 02:33:31 +01:00
Сергей Прохоров
9a735bf928
Make it possible to change logick of test server RPC handling 2019-03-07 02:32:22 +01:00
Сергей Прохоров
321cfd69fb
Add some more metric-based asserts to CT 2019-03-05 01:52:40 +01:00
Сергей Прохоров
40dfa84d61
Add some docstrings to tests 2019-03-05 01:30:38 +01:00
Сергей Прохоров
eea142f6fe
Add property-based tests to mtp_codec 2019-03-05 01:30:02 +01:00
Сергей Прохоров
e9ece4c817
Split mtp_test_middle_server to 2 modules
- one with just middle-server implementation
- one with proxy config mock and multi-server DC setup
2019-03-04 11:15:07 +01:00
Сергей Прохоров
8a80fd4b59
Fix common test (webserver root dir) 2019-03-04 11:07:42 +01:00
Сергей Прохоров
8be2a3da0e
Add test proxy client and some basic common-tests 2019-03-04 01:44:47 +01:00