# Architecture Overview

@mataram/wa is a TypeScript WebSocket client that speaks the WhatsApp Web binary
protocol directly — no browser, no Selenium. This document explains how the
major subsystems work.

---

## 1. Connection & Noise Protocol Handshake

WhatsApp Web uses the **Noise Protocol Framework** with the **XXpsk0** pattern
to establish an encrypted tunnel between the client and the server. (The
constant `NOISE_MODE = 'Noise_XX_25519_AESGCM_SHA256'` in
`src/Defaults/index.ts` defines the handshake shape.)

### Handshake flow

```
Client                             Server
  |                                   |
  |  ClientHello (ephemeral public)   |
  |---------------------------------->|
  |  ServerHello  (ephemeral, static, |
  |   payload = cert chain)           |
  |<----------------------------------|
  |  ClientFinish  (static encrypted, |
  |   encrypted payload)              |
  |---------------------------------->|
  |  Transport (AES-256-GCM)          |
  |<=================================>|
```

1. **ClientHello**: The client generates an ephemeral Curve25519 key pair
   (`ephemeralKeyPair` in `socket.ts:121`) and sends the public half as a
   `proto.HandshakeMessage.clientHello`.

2. **ServerHello**: The server replies with its own ephemeral, a static key
   (encrypted), and a certificate chain (`proto.CertChain`). The handshake
   handler in `noise-handler.ts`:
   - Mixes the shared secret into the hash via `mixIntoKey`
   - Verifies the server certificate against WA's well-known public key and
     serial number
   - Derives `encKey`/`decKey` via HKDF

3. **ClientFinish**: The client encrypts its own noise static key
   (`creds.noiseKey.public`) with the current cipher state, then sends a
   protobuf `ClientPayload` (registration or login node) encrypted under the
   handshake cipher.

4. **Transport ready**: `noise.finishInit()` derives transport keys and
   switches the cipher to a `TransportState` with monotonic AEAD counters
   (separate read/write counters).

### Frame encoding

Every message on the wire is framed as a 3-byte big-endian length prefix
followed by the payload. The first frame carries an intro header
(`NOISE_WA_HEADER = [87, 65, 6, DICT_VERSION]`), optionally prefixed with
routing info (`ED` prefix) if `creds.routingInfo` is set.

## 2. Socket Layer Architecture

The socket is built as a chain of composable layers, each adding functionality:

```
makeWASocket
  └─ makeCommunitiesSocket
       └─ makeBusinessSocket
            └─ makeMessagesRecvSocket
                 └─ makeMessagesSocket
                      └─ makeNewsletterSocket
                           └─ makeGroupsSocket
                                └─ makeChatsSocket
                                     └─ makeSocket (base)
```

The entry point `makeWASocket` (in `Socket/index.ts`) simply calls
`makeCommunitiesSocket` with merged config. Each layer receives the return
value of the previous layer and extends it using object spread:

```ts
return {
  ...sock,          // everything from layers below
  relayMessage,     // new method
  sendMessage,      // new method
  sendReceipt,      // new method
  // ...
}
```

### Layer responsibilities

| Layer | File | Adds |
|---|---|---|
| `makeSocket` | `socket.ts` | WebSocket connection, Noise handshake, binary node send/recv, `query()` (IQ send+wait), keep-alive, QR/pairing flow, pre-key upload, USync queries, WAM buffer |
| `makeChatsSocket` | `chats.ts` | Chat mutation (archive, pin, mute, mark read), presence, privacy settings, app state sync resync |
| `makeGroupsSocket` | `groups.ts` | Group metadata, participant management, group invite, ephemeral settings |
| `makeNewsletterSocket` | `newsletter.ts` | Newsletter subscribe/unsubscribe, message fetching |
| `makeMessagesSocket` | `messages-send.ts` | `sendMessage()`, `relayMessage()`, media upload, message retry cache, USync device discovery, message encryption, tctoken issuance |
| `makeMessagesRecvSocket` | `messages-recv.ts` | Message/notification receipt, decryption + decoding, retry handling, identity change handling, Mex notifications |
| `makeBusinessSocket` | `business.ts` | Business profile, product catalog |
| `makeCommunitiesSocket` | `communities.ts` | Community parent-group management |

### Why layered?

- **Separation of concerns** — crypto/connection logic doesn't mix with
  message formatting or group management.
- **Tree-shakeable** — downstream consumers that only need the base socket
  avoid importing group or newsletter code.
- **Testability** — each layer can be mocked at the boundary below it.

## 3. Binary Nodes

WhatsApp's wire protocol is built on **binary XML** — a compact tokenized
format that replaces verbose XML with dictionaries of single-byte and
double-byte tokens. The type is defined in `WABinary/types.ts`:

```ts
type BinaryNode = {
  tag: string
  attrs: { [key: string]: string }
  content?: BinaryNode[] | string | Uint8Array
}
```

A binary node is the equivalent of an XML element:
- `tag` is the element name (e.g., `'iq'`, `'message'`, `'receipt'`)
- `attrs` are the XML attributes (e.g., `{ to: 's.whatsapp.net', type: 'get', id: 'abc123' }`)
- `content` is either child elements, a raw string, or a binary blob

### Token dictionary

The token system (`WABinary/constants.ts`) maps commonly repeated strings to
single bytes (positions 0-255 in `SINGLE_BYTE_TOKENS`) or double bytes (four
dictionaries of ~250 entries each). This dramatically reduces message size.

Example tokens: `'iq'` = byte 26, `'message'` = byte 20, `'s.whatsapp.net'` =
byte 3, `'enc'` = byte 28.

### CB routing system

When a binary node arrives, `socket.ts:onMessageReceived` generates event
names by pattern-matching the tag and attributes:

```ts
const l0 = frame.tag
const l1 = frame.attrs || {}
const l2 = frame.content?.[0]?.tag || ''

for (const key of Object.keys(l1)) {
  ws.emit(`CB:${l0},${key}:${l1[key]},${l2}`, frame)
  ws.emit(`CB:${l0},${key}:${l1[key]}`, frame)
  ws.emit(`CB:${l0},${key}`, frame)
}
ws.emit(`CB:${l0},,${l2}`, frame)
ws.emit(`CB:${l0}`, frame)
```

This means handlers register on patterns like:

```ts
// Match <notification type="w:gp2"> with any child
ws.on('CB:notification,type:w:gp2', handler)

// Match <ib> with <offline> child
ws.on('CB:ib,,offline', handler)

// Match <stream:error>
ws.on('CB:stream:error', handler)
```

The `DEF_CALLBACK_PREFIX` is `'CB:'` and the `DEF_TAG_PREFIX` is `'TAG:'` —
TAG events are used for IQ responses matched by message ID.

### Binary encoding

Encoding/decoding lives in `WABinary/encode.ts` and `WABinary/decode.ts`. The
encoder writes a stream of tokens, strings, integers, and nested nodes using
the dictionary. Decoding reverses this — it reads bytes, looks up tokens, and
reconstructs the `BinaryNode` tree.

## 4. Event System

### MataramEventEmitter

The event system (`Types/Events.ts`) defines a strongly-typed event map:

```ts
type MataramEventMap = {
  'connection.update': Partial<ConnectionState>
  'creds.update': Partial<AuthenticationCreds>
  'messages.upsert': { messages: WAMessage[]; type: MessageUpsertType }
  'chats.upsert': Chat[]
  'chats.update': ChatUpdate[]
  'group-participants.update': { id: string; action: ParticipantAction; participants: string[] }
  // ... many more
}
```

The emitter interface (`MataramEventEmitter`) mirrors Node's EventEmitter but
with typed events:

```ts
interface MataramEventEmitter {
  on<T extends keyof MataramEventMap>(event: T, listener: (arg: MataramEventMap[T]) => void): void
  off<T extends keyof MataramEventMap>(event: T, listener: (arg: MataramEventMap[T]) => void): void
  emit<T extends keyof MataramEventMap>(event: T, arg: MataramEventMap[T]): boolean
}
```

### Event buffering (makeEventBuffer)

The `makeEventBuffer` (`Utils/event-buffer.ts`) wraps a raw EventEmitter and
adds **buffering** for a well-known set of events. When buffering is active,
incoming events are accumulated into a single consolidated data structure
rather than emitted immediately.

Bufferable events include: `'messaging-history.set'`, `'chats.upsert'`,
`'chats.update'`, `'chats.delete'`, `'contacts.upsert'`,
`'contacts.update'`, `'messages.upsert'`, `'messages.update'`,
`'messages.delete'`, `'messages.reaction'`, `'message-receipt.update'`,
`'groups.update'`.

### The process() pattern

Instead of registering separate handlers per event, consumers can use
`process()` to receive a batch of consolidated events:

```ts
const unlisten = ev.process(async (events) => {
  if (events['messages.upsert']) {
    // handle all upserted messages in one call
  }
  if (events['chats.update']) {
    // handle all chat updates in one call
  }
})
```

This is used internally during initial sync to avoid thousands of individual
event emissions.

### Buffer lifecycle

1. `ev.buffer()` — starts buffering (called on login)
2. During buffering, events accumulate in the `BufferedEventData` struct
3. `ev.flush()` — atomically consolidates and emits all buffered events
4. The buffer is flushed automatically when `CB:ib,,offline` is received
   (meaning all offline notifications have been processed)

## 5. Auth State

### AuthenticationCreds

The credentials object (`Types/Auth.ts:AuthenticationCreds`) holds all
persistent authentication and cryptographic material:

```ts
type AuthenticationCreds = {
  // Noise handshake key (ephemeral per-connection key is generated separately)
  noiseKey: KeyPair
  // Key used to encrypt the pairing code exchange
  pairingEphemeralKeyPair: KeyPair
  // ADv secret used to verify account signature
  advSecretKey: string
  // The long-term identity key (Curve25519)
  signedIdentityKey: KeyPair
  // The signed pre-key (medium-term, rotated on server request)
  signedPreKey: SignedKeyPair
  // Random registration ID (client-specified, server-validated)
  registrationId: number
  // Pre-key tracking
  nextPreKeyId: number
  firstUnuploadedPreKeyId: number
  // Identity
  me?: Contact         // { id, name, lid }
  account?: proto.IADVSignedDeviceIdentity
  signalIdentities?: SignalIdentity[]
  // App state
  myAppStateKeyId?: string
  // History sync
  processedHistoryMessages: MinimalMessage[]
  accountSyncCounter: number
  // Settings
  accountSettings: AccountSettings
  registered: boolean
  pairingCode: string | undefined
  routingInfo: Buffer | undefined
}
```

### SignalKeyStore interface

The key store provides the persistence layer for Signal protocol data:

```ts
type SignalKeyStore = {
  get<T extends keyof SignalDataTypeMap>(
    type: T,
    ids: string[]
  ): Awaitable<{ [id: string]: SignalDataTypeMap[T] }>

  set(data: SignalDataSet): Awaitable<void>

  clear?(): Awaitable<void>
}
```

Where `SignalDataTypeMap` defines the stored types:

| Key | Value |
|---|---|
| `'pre-key'` | `KeyPair` |
| `'session'` | `Uint8Array` (serialized Signal session) |
| `'sender-key'` | `Uint8Array` (serialized sender key record) |
| `'sender-key-memory'` | `{ [jid: string]: boolean }` |
| `'app-state-sync-key'` | `proto.Message.IAppStateSyncKeyData` |
| `'app-state-sync-version'` | `LTHashState` |
| `'lid-mapping'` | `string` (PN user → LID user mapping) |
| `'device-list'` | `string[]` (device IDs for a user) |
| `'tctoken'` | `{ token: Buffer; timestamp?: string }` |
| `'identity-key'` | `Uint8Array` |

### Transaction capability

The `addTransactionCapability` function (`Utils/auth-utils.ts`) wraps a
`SignalKeyStore` with DB-like transaction semantics using
`AsyncLocalStorage`:

```ts
const keys = addTransactionCapability(authState.keys, logger, transactionOpts)

await keys.transaction(async () => {
  // All reads are cached in-memory
  // All writes are batched as "mutations"
  // On success, mutations are committed atomically
  // On error, mutations are discarded
}, 'transaction-key')
```

Transactions provide:
- **Atomic commits** — writes are batched and committed in a single
  `keys.set()` call, with retry logic
- **Read-caching** — within a transaction, reads are served from an in-memory
  cache; repeated reads of the same key avoid database lookups
- **Nested transaction support** — nested `transaction()` calls reuse the
  outer context
- **Per-key-type isolation** — pre-key deletions have special validation to
  prevent deleting non-existent keys

## 6. Message Flow (Sending)

The sending pipeline begins at `sendMessage()` in `messages-send.ts`.

### Content type dispatch

`sendMessage()` is a large function that inspects the `AnyMessageContent` and
dispatches to the appropriate handler:

```ts
if (content.rich && content.text) {
  // Rich text — parse markdown/formatting, build rich message content
}
if (content.buttons) {
  // Interactive buttons — converts to native flow buttons
}
if (content.templateButtons) {
  // Template buttons — converts URL/call/quick_reply buttons
}
if (content.sections) {
  // List message — converts to single_select native flow
}
if (content.disappearingMessagesInChat) {
  // Group ephemeral setting
}
// Default: text, media, polls, events, edits, deletes, pins
```

For the default path (text/media/polls/etc.), `generateWAMessage()` constructs
the `WAMessage` protobuf, optionally uploading media and fetching link
previews.

### relayMessage — encrypt + send

The core send function is `relayMessage()` (line 618), which:

1. **Determines recipients**: For groups, fetches group metadata and
   participant devices via USync. For 1:1, enumerates devices for both
   parties.

2. **Asserts sessions**: Calls `assertSessions()` to ensure Signal sessions
   exist with all recipients, auto-fetching pre-keys from the server if
   needed.

3. **Encrypts**:
   - **Group**: `signalRepository.encryptGroupMessage()` — uses sender key
     encryption (one ciphertext for all group members). A
     `SenderKeyDistributionMessage` (SKDM) is sent to members who haven't
     received the sender key yet.
   - **1:1**: `signalRepository.encryptMessage()` — per-device session
     encryption. Own devices receive the message wrapped in a
     `deviceSentMessage` (DSM).

4. **Attaches extras**: device identity (for new sessions), message reporting
   token, tctoken (privacy token for 1:1 chats).

5. **Builds the stanza**: constructs a `<message>` binary node with `<enc>`
   children and optional `<participants>`, `<device-identity>`,
   `<tctoken>`, `<reporting>`.

6. **Sends**: `sendNode(stanza)` encodes the binary node and sends it through
   the Noise transport.

7. **Post-send**: Issues a new tctoken to the contact (if needed) and adds
   the message to the retry cache.

## 7. Message Receiving

The receive path starts in `messages-recv.ts` when a `<message>` stanza is
received via the `CB:message` handler.

### Decryption & decoding

1. **decodeMessageNode** (`Utils/decode-wa-message.ts`): parses the stanza's
   attributes to determine message type (chat/group/broadcast/newsletter),
   chat ID, author, `fromMe` status, and addressing mode (PN vs LID).

2. **decrypt()**: For each `<enc>` child in the stanza:
   - **`pkmsg`** (pre-key message): first message to a new session —
     `repository.decryptMessage()` with type `'pkmsg'`
   - **`msg`** (normal message): subsequent messages in an established
     session — `repository.decryptMessage()` with type `'msg'`
   - **`skmsg`** (sender key message): group messages —
     `repository.decryptGroupMessage()`
   - **`plaintext`**: newsletter messages (no encryption)

3. **Message decoding**: The decrypted binary is unpadded (random max-16
   padding is stripped) and decoded as `proto.Message`. If the message
   contains a `SenderKeyDistributionMessage`, it's processed to establish
   group session state.

4. **Identity change detection**: During PKMSG decryption, the sender's
   identity key is extracted and compared against stored identity. If
   changed, the session is cleared and re-established.

### Event emission

Once decrypted, the message goes through `processMessage()` in
`process-message.ts`, which handles:

- **Protocol messages** (history sync, app state key share, message edits,
  revokes, ephemeral settings, LID migration, etc.)
- **Reactions** → `'messages.reaction'` event
- **Stub messages** (group participant changes, group settings changes,
  etc.) → appropriate events
- **Real messages** → `'messages.upsert'` event, with chat unread count
  increment

### NACK flow

If decryption fails, the client sends a NACK (negative acknowledgment) with a
specific reason code:

```ts
sendMessageAck(node, NACK_REASONS.SignalErrorOldCounter)     // 496
sendMessageAck(node, NACK_REASONS.MissingMessageSecret)       // 495
```

The server may respond with a retry receipt, triggering the message retry
system (see Signal doc §9).

## 8. App State Sync

WhatsApp uses a **collection-based sync** system for replicating client-side
state (settings, blocklist, labels, pinned chats, etc.).

### Collections

App state is organized into patches with different sync priorities:

| Patch name | Priority | Contents |
|---|---|---|
| `critical_block` | Highest | Must be synced before anything else |
| `critical_unblock_low` | High | Almost as critical |
| `regular` | Normal | Standard settings |
| `regular_low` | Low | Non-urgent |
| `regular_high` | Higher than regular | Important but not critical |

### Sync mechanism

1. On connection, the server sends a `server_sync` notification for any
   collection that has pending changes.
2. `resyncAppState()` fetches the patch from the server (an IQ with
   `xmlns='w:sync:app:state'`).
3. The patch contains a list of sync actions, each with:
   - An operation type (`SET`, `REMOVE`)
   - The action data (protobuf-encoded `SyncActionValue`)
   - A version hash for integrity verification
4. Actions are processed by type — contact actions, chat actions, setting
   actions, etc.
5. Version state is persisted as `LTHashState` (Merkle-tree-style integrity
   hash).

### Initial sync

History sync (`messaging-history.set` event) is driven by
`ProtocolMessage.Type.HISTORY_SYNC_NOTIFICATION`, which triggers downloading
and processing of a `HistorySyncNotification` protobuf containing chats,
messages, and contacts in bulk.

## 9. WAM Analytics (BinaryInfo)

The `WAM/` directory implements WhatsApp's telemetry system. `BinaryInfo`
(`WAM/BinaryInfo.ts`) is a buffer that accumulates analytics events:

```ts
class BinaryInfo {
  protocolVersion = 5
  sequence = 0
  events = [] as EventInputType[]
  buffer: Buffer[] = []
}
```

Events are collected during a session and periodically sent to the server via
`sendWAMBuffer()`, which wraps them in an IQ with `xmlns='w:stats'`. This
provides WhatsApp with usage metrics (message counts, connection quality,
feature usage, etc.).

The `publicWAMBuffer` is accessible on the socket object, allowing downstream
code to append custom analytics events.

## 10. USync

USync is WhatsApp's user discovery protocol. It queries the server for
information about users in batch.

### Query structure

A USync query (`WAUSync/USyncQuery.ts`) consists of:

```ts
const query = new USyncQuery()
  .withContext('interactive')     // context: 'background' | 'interactive' | 'message'
  .withUser(new USyncUser().withId('123@s.whatsapp.net'))
  .withDeviceProtocol()           // fetch device list
  .withLIDProtocol()              // fetch LID mapping
  .withContactProtocol()          // fetch contact status
  .withStatusProtocol()           // fetch status
  .withDisappearingModeProtocol() // fetch disappearing mode
  .withUsernameProtocol()         // fetch username
  .withBotProfileProtocol()       // fetch bot profile
```

The query is sent as an IQ to `s.whatsapp.net` with `xmlns='usync'`. The
response contains a `<list>` of `<user>` nodes, each with child elements
corresponding to the requested protocols.

### Protocols

| Protocol | What it returns |
|---|---|
| `USyncDeviceProtocol` | Device list (`device:0`, `device:1`, etc.) with key indices |
| `USyncLIDProtocol` | LID (stable internal ID) for a given phone number |
| `USyncContactProtocol` | Whether the user exists on WhatsApp |
| `USyncStatusProtocol` | User's status message |
| `USyncDisappearingModeProtocol` | User's disappearing message settings |
| `USyncUsernameProtocol` | Username |
| `USyncBotProfileProtocol` | Bot profile info |

### Key uses

- **Device discovery**: Before sending a 1:1 message, the library fetches all
  devices for both sender and recipient via `getUSyncDevices()`. This
  ensures messages reach all linked devices.

- **LID resolution**: When a message arrives with a PN JID, the library may
  need the LID (stable internal ID) for session lookup. USync LID protocol
  resolves this mapping.

- **onWhatsApp**: Checks if a phone number is registered on WhatsApp by
  querying the contact protocol.

- **LID batch resolution**: The `pnFromLIDUSync()` function fetches LID
  mappings for multiple PNs at once, used during history sync to migrate
  sessions from PN-based to LID-based addressing.