File transfer pipeline

Everything is parts#

MTProto has no streaming file primitive. A file is split into fixed-size parts — up to 512 KB each — and every part is its own RPC: upload.saveFilePart for files up to 10 MB, upload.saveBigFilePart (with a total-parts count) above that. Downloads mirror it: upload.getFile with an offset and limit per request. The commit step is atomic — the media message referencing the upload only sends after every part is stored.

upload:   sendFile("./video.mp4")
          1. split into parts of at most 512 KB
          2. per part:  upload.saveFilePart      (files <= 10 MB)
                        upload.saveBigFilePart   (files >  10 MB)
          3. commit:    messages.sendMedia -- only after every part is stored

download: downloadMedia(message)
          1. per part:  upload.getFile { offset, limit }
                        (parallel in-flight requests, sequential offsets)
          2. CDN case:  upload.getCdnFile, hashes checked against master DC
          3. parts reassembled -> Buffer, or flushed to outputFile

Files don't necessarily live on your home DC. FileMigrateError tells the client which DC holds the bytes; teleproto dials it (a media lane with its own temp auth key) and retries there. Popular public media may come from CDN DCs via upload.getCdnFile, with hashes verified against the master DC.

The session pool#

Since v1.227.4 uploads, downloads, and API calls draw from a single per-DC session poolinstead of three separate ones. Pool sessions to the same DC share that DC's permanent auth key and server salt — a new lane is a TCP connect plus a temp-key bind, not a full DH handshake. TCP connects are staggered rather than fired simultaneously, which is what fixed the reconnect storm on accounts whose home DC also stores their files.

Senders are leasedfrom the pool with explicit enter/leave tracking (v1.227.3). Before that, the idle-timeout reaper could disconnect a sender in the middle of a long upload it didn't know was using it — the "big uploads die at 90%" bug. A leased sender can't be reaped.

Uploads: sliding window#

Old scheme: send parts in batches of N, wait for the whole batch, send the next. One slow part — or one part answering FLOOD_WAIT — stalled its entire batch. Since v1.227.3 uploads keep a sliding window of in-flight parts: each acknowledged part immediately admits the next, so a hiccup costs one slot, not the pipeline. The workers option is the window width.

Part payloads take the single-copy path from v1.228 — read from disk, encrypted in place, written to the socket. On a fast link the practical ceiling is Telegram's per-connection throttling, which is why more workers (more parallel parts) beats any amount of per-part tuning.

Downloads: scheduler & warm lanes#

All downloads go through one scheduler (v1.227.4) that balances parts across the pool with an adaptive per-DC policy — a DC answering slowly gets fewer in-flight parts, a fast one gets more. Each part request carries a deadline (requestDeadlineMs): a part that blows it is re-queued on another sender instead of blocking the file. Sizes are trusted end-to-end from document.size, so a short intermediate response triggers a retry, not a silently truncated output file.

After a download finishes, its media sessions stay connected for 60 seconds (v1.228). Bursty workloads — a bot that downloads every photo in a channel — pay the media-lane setup once per burst, not once per file.

Diagnosing slow transfers#

const client = new TelegramClient(session, apiId, apiHash, {
  connectionRetries: 5,

  // per-call chunk parallelism for uploads
  // (sendFile / uploadFile also take { workers })

  maxConcurrentDownloads: 6,   // client-wide ceiling on parallel chunks

  downloadPool: {
    poolSize: 4,               // parallel senders the pool keeps hot
    workers: 8,                // in-flight parts per sender
    requestDeadlineMs: 30_000, // per-part deadline; raise on slow links
  },
});

• Single big file, fast network, slow transfer — raise workers (window width). Telegram throttles per connection; parallelism is the lever.
• Many small files — raise poolSize. Small files are latency-bound; more lanes hide the round-trips.
• Parts timing out on mobile / high-latency links — raise requestDeadlineMs. The default assumes datacenter-grade latency; on a 3G hop the deadline fires before the bytes arrive and the re-queue makes it worse.
• Everything slow, CPU pinned— you're on a teleproto older than 1.227.3; the AES-IGE context reuse and native-CBC decrypt landed there. Update before tuning anything.