Fix Fake-TLS (EE mode) rejected when domain option specified

[Copilot]

When -D domain option is used to enable Fake-TLS mode, connections using EE mode (tag 0xeeeeeeee) were incorrectly rejected with "Expected random padding mode" error. This prevented the documented EE mode secret format (ee + secret + domain_hex) from working.

Changes

• Refined validation in net/net-tcp-rpc-ext-server.c (lines 1318-1321) to properly allow EE/EF modes and reject DD mode when allow_only_tls flag is set

The original buggy check was:

if (tag != 0xdddddddd && allow_only_tls) {
  vkprintf (1, "Expected random padding mode\n");
  RETURN_TLS_ERROR(default_domain_info);
}

This incorrectly rejected EE/EF modes (Fake-TLS) and only allowed DD mode (random padding only) when -D was used.

The fixed validation is:

if (tag == 0xdddddddd && allow_only_tls) {
  vkprintf (1, "Expected Fake-TLS mode (EE/EF), got random padding mode (DD)\n");
  RETURN_TLS_ERROR(default_domain_info);
}

This properly rejects DD mode and allows EE/EF modes when -D is specified, maintaining security validation while fixing the bug.

Behavior

When -D option is used:

• ✅ EE mode (0xeeeeeeee - Fake-TLS + Padding) is allowed
• ✅ EF mode (0xefefefef - Fake-TLS compact) is allowed
• ❌ DD mode (0xdddddddd - random padding only) is rejected with clear error message

The logic correctly separates two orthogonal concerns:

• -D option sets allow_only_tls to enforce Fake-TLS transport modes (EE/EF)
• -R option sets ext_rand_pad_only to restrict to DD mode only

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Fake TSL not working</issue_title>
<issue_description>Hi!

I've tried to setup Fake-TLS configuration but it doesn't work in my case.

Stept to reproduce:

1. Clean Ubuntu 24.04 (LTS) + update with all latest packages with apt-get update && apt-get upgrade
2. Clone repo
3. Install dependencies with apt install git curl build-essential libssl-dev zlib1g-dev
4. Build with make && cd objs/bin
5. Generate secret with head -c 16 /dev/urandom | xxd -ps and domain hex with echo -n www.google.com | xxd -plain
6. Run with ./mtproto-proxy -u nobody -p 8888 -H 443 -S <generated secret> -D www.google.com --http-stats --aes-pwd proxy-secret proxy-multi.conf -M 1
7. Connect in Telegram app with secret ee + <generated secret> + 7777772e676f6f676c652e636f6d

Telegram client (macOS 12.3.277495 Stable or iOS 12.2.4) cannot connect to proxy.
Same build but without -D www.google.com and standard secret in app setting works perfectly fine.

Any ideas what is the problem?

Included in the attachment log with -v flag

mtproto-verbose.log

</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Fake TSL not working #23

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[dvershinin]

@copilot why remove the validation entirely? Perhaps just relaxing it a bit is a better approach.

[Copilot]

@copilot why remove the validation entirely? Perhaps just relaxing it a bit is a better approach.

You're right. I've refined the validation instead of removing it entirely. The new check inverts the logic to properly reject DD mode (random padding only) when -D is used, while allowing EE/EF modes (Fake-TLS). This maintains security validation while fixing the bug. See commit 926ec36.

[dvershinin]

@copilot still not working, still same behavior as in https://github.com/user-attachments/files/24359663/mtproto-verbose.log

[dvershinin]

@copilot hello