Sync Protocol

Overview

The sync engine in the Headless Go client for Obsidian communicates with Obsidian's cloud servers over a WebSocket (WSS) connection.

Connection

Transport

Heartbeat

To detect dead connections, the client maintains a heartbeat:

The client sends:

{ "op": "ping" }

And expects the server to respond with:

{ "op": "pong" }

Connection Lifecycle

1. Init Request (Client → Server)

The client opens the connection and sends an init message to authenticate and start the sync session:

{
  "op": "init",
  "token": "<auth-token>",
  "id": "<vault-id>",
  "keyhash": "<key-hash>",
  "version": 0,
  "initial": true,
  "device": "My Device",
  "encryption_version": 3
}

2. Init Response (Server → Client)

On success:

{
  "res": "ok",
  "user_id": 12345,
  "max_size": 208666624
}

On error:

{
  "res": "err",
  "msg": "Authentication failed."
}

3. Synchronous Handshake

After the initial {"res":"ok"} response, the server performs a synchronous handshake before entering the normal read loop:

1. Client sends init request
2. Server responds with {"res":"ok", ...} (or error)
3. Server sends all existing file records as push messages, including deleted files (deleted: true)
4. Server sends {"op":"ready","version":N} to signal completion

4. Ready Notification

Once the handshake completes, the server sends:

{
  "op": "ready",
  "version": 150
}

The version number should be stored and sent in the next init to resume from this point.

Protocol Operations

Push

Push Request (Client → Server)

{
  "op": "push",
  "path": "<encrypted-path>",
  "relatedpath": "<encrypted-related-path>",
  "extension": ".md",
  "hash": "<encrypted-hash>",
  "ctime": 1700000000000,
  "mtime": 1700000000000,
  "folder": false,
  "deleted": false,
  "size": 1024,
  "pieces": 1
}

Push Response

If the server already has the content (deduplication):

{ "res": "ok" }

If the server needs the content:

{ "res": "next" }

After "next", the client sends binary chunks (max 2 MB each) as binary frames. After each chunk, the server responds with another "next" or, on the final chunk:

{ "res": "ok" }

Server Push Notification (Server → Client)

When any device pushes a change, the server broadcasts a push notification to all connected clients on the vault:

{
  "op": "push",
  "path": "<encrypted-path>",
  "hash": "<encrypted-hash>",
  "ctime": 1700000000000,
  "mtime": 1700000000000,
  "size": 1024,
  "folder": false,
  "deleted": false,
  "uid": 42,
  "device": "Other Device",
  "user": "user@example.com"
}

Pull

Pull Request (Client → Server)

{
  "op": "pull",
  "uid": 42
}

Pull Response

{
  "res": "ok",
  "size": 1024,
  "pieces": 1,
  "deleted": false,
  "hash": "<encrypted-or-raw-hash>"
}

If deleted is true, no binary data follows. Otherwise, the server sends pieces binary frames containing the encrypted file content, which the client concatenates and decrypts.

Deleted Files

Request a list of deleted files:

// Request
{ "op": "deleted", "suppressrenames": true }

// Response
{
  "res": "ok",
  "items": [
    {
      "path": "<encrypted-path>",
      "hash": "...",
      "ctime": 1700000000000,
      "mtime": 1700000000000,
      "size": 0,
      "folder": false,
      "deleted": true,
      "uid": 43,
      "device": "...",
      "user": "..."
    }
  ]
}

History

Request version history for a file:

// Request
{ "op": "history", "path": "<encrypted-path>", "last": 0 }

// Response
{
  "res": "ok",
  "items": [
    {
      "path": "<encrypted-path>",
      "hash": "...",
      "ctime": 1700000000000,
      "mtime": 1700000000000,
      "size": 1024,
      "folder": false,
      "deleted": false,
      "uid": 41,
      "device": "...",
      "user": "..."
    }
  ]
}

Size

Request vault storage usage:

// Request
{ "op": "size" }

// Response
{ "res": "ok", "size": 52428800 }

Restore

Restore a deleted file by UID:

// Request
{ "op": "restore", "uid": 43 }

// Response
{ "res": "ok" }

Binary Data Streaming

File content is streamed as binary WebSocket frames in chunks of up to 2 MB (2,097,152 bytes). The number of chunks is communicated in the pieces field.

Push Flow

1. Client sends push metadata (JSON)
2. If server responds { "res": "next" }, client streams chunks
3. Each chunk gets a JSON response
4. Final chunk gets { "res": "ok" }

Pull Flow

1. Client sends pull request (JSON)
2. Server responds with metadata including pieces count
3. Server sends pieces binary frames
4. Client concatenates and decrypts

Parallel Downloads

The pull protocol has no request ID field — each pull request expects an immediate response. This means concurrent pulls on a single WebSocket connection are not possible because responses would interleave unpredictably.

To parallelize downloads, the sync client opens multiple WebSocket connections, one per worker. Each connection completes its own init handshake independently. Workers then pull files sequentially on their own connection.

Version Tracking & Conflict Resolution

Version Numbers

The server assigns a monotonically increasing version number to every change in a vault. When a client connects, it sends the last known version in the init request. The server then streams all changes with a higher version during the handshake.

{
  "op": "ready",
  "version": 150
}

Conflict Resolution

The Obsidian sync protocol uses a last-write-wins strategy based on mtime (modification time). When a client receives a server push notification for a file it has also modified locally, it compares mtime values:

• If the server's mtime is newer, the client accepts the server version and overwrites the local file.
• If the local mtime is newer, the client pushes its local version to the server.

Because the server echoes push notifications, the client must ignore self-echoes (matching device and mtime) to avoid ping-ponging the same change indefinitely.

Error Handling

Server errors use one of two formats:

{ "res": "err", "msg": "Human-readable error message" }

Or (observed during init):

{ "status": "err", "message": "Human-readable error message" }

The client should check both res and status fields for errors.

Reconnection Strategy

The client uses exponential backoff for reconnection:

Path Encoding

All file paths transmitted over the wire are encrypted using the vault's encryption provider:

• V0: Paths are base64-encoded then AES-GCM encrypted
• V2/V3: Paths are AES-SIV encrypted (deterministic, allowing server-side dedup)

The encryption is applied to the full path string, so / directory separators are encrypted as part of the path rather than preserved on the wire. See Encryption Protocol for details.