Federico MeiniReal-time systems, AI agents & more
IT
Work with me(opens in a new tab)
← All posts

Building a reliable WhatsApp Cloud API webhook receiver at scale

How to build a WhatsApp Cloud API webhook receiver that holds up: signature checks on the raw body, fast 200s, dedup, per-contact ordering and media handling.

By 7 min readLeggi in italiano

A reliable WhatsApp Cloud API webhook receiver does four things in the request path and nothing else: it verifies the X-Hub-Signature-256 header against the raw request body, persists the payload durably, responds with a 200 quickly, and hands the work to asynchronous processing. Everything that makes WhatsApp integrations hard (deduplicating at-least-once deliveries, keeping each conversation in order, reconciling out-of-order status updates, respecting the 24-hour window, downloading media) happens after the 200, where a slow step can’t cause Meta to retry and multiply your load. I’ve spent six years at Turn.io working on a WhatsApp platform that handles millions of messages a day, and this is how I’d structure the receiver in Elixir and Phoenix.

TL;DR

  • Keep the raw body around and verify X-Hub-Signature-256 (HMAC-SHA256 with your app secret) over those exact bytes.
  • In the request: verify, persist, return 200. Return a 5xx only if you couldn’t persist, so Meta retries.
  • Webhooks are at-least-once: deduplicate inbound messages by their message id with a unique index.
  • Process each contact’s messages in order, one at a time: one process per contact, or a queue partitioned by contact.
  • Status updates arrive out of order: only ever move a message’s status forward.
  • Track the 24-hour customer service window per contact and switch to templates outside it.
  • Download media promptly in a background job: the download URL is short-lived.

What Meta sends you

Webhook payloads are batched. A single POST contains an entry array, each entry has changes, and each change has a value that can hold messages (inbound messages from users), statuses (updates about messages you sent), contacts, and metadata with the phone_number_id the event belongs to. Don’t assume one message per request, and don’t assume a request contains only one kind of event.

Before any of that, Meta verifies your endpoint with a GET request carrying hub.mode=subscribe, a hub.verify_token you chose, and a hub.challenge you must echo back. The verify token is only used for this handshake. The app secret is what signs every POST. They are different values, and mixing them up is a classic first-day bug.

Verifying X-Hub-Signature-256 over the raw body

The signature is an HMAC-SHA256 of the request body using your app secret, sent as sha256=<hex digest>. It’s computed over the exact bytes Meta sent. By the time a Phoenix controller runs, Plug.Parsers has consumed the body and decoded the JSON, and re-encoding it won’t reproduce the same bytes. So keep a copy while parsing, with a custom body reader:

defmodule MyAppWeb.CacheRawBody do
  @moduledoc "Keeps the raw request body for webhook signature verification."

  @webhook_path "/webhooks/whatsapp"

  def read_body(conn, opts) do
    case Plug.Conn.read_body(conn, opts) do
      {:ok, chunk, conn} -> {:ok, chunk, store(conn, chunk)}
      {:more, chunk, conn} -> {:more, chunk, store(conn, chunk)}
      {:error, _reason} = error -> error
    end
  end

  # Only keep a copy for the webhook route, so other requests don't pay for it.
  defp store(%Plug.Conn{request_path: @webhook_path} = conn, chunk) do
    Plug.Conn.assign(conn, :raw_body, [conn.assigns[:raw_body] || "", chunk])
  end

  defp store(conn, _chunk), do: conn
end

Wire it into the parsers in your endpoint:

plug Plug.Parsers,
  parsers: [:urlencoded, :multipart, :json],
  pass: ["*/*"],
  body_reader: {MyAppWeb.CacheRawBody, :read_body, []},
  json_decoder: Phoenix.json_library()

Then verify in a plug on the webhook pipeline, using a constant-time comparison:

defmodule MyAppWeb.Plugs.VerifyWhatsAppSignature do
  import Plug.Conn

  def init(opts), do: opts

  def call(conn, _opts) do
    secret = Application.fetch_env!(:my_app, :whatsapp_app_secret)
    raw_body = IO.iodata_to_binary(conn.assigns[:raw_body] || "")
    digest = :crypto.mac(:hmac, :sha256, secret, raw_body) |> Base.encode16(case: :lower)

    with [signature] <- get_req_header(conn, "x-hub-signature-256"),
         true <- Plug.Crypto.secure_compare("sha256=" <> digest, String.downcase(signature)) do
      conn
    else
      _ -> conn |> send_resp(401, "invalid signature") |> halt()
    end
  end
end

If signatures fail in production but pass locally, look for something between Meta and your app that rewrites the body: a proxy that decompresses or re-encodes JSON, or middleware that parses the body before your reader sees it.

Respond fast, process later

Meta retries deliveries that fail or time out, with decreasing frequency, for up to several days according to its documentation. That’s good for durability and bad for load: if your handler is slow because a downstream dependency is slow, requests time out, Meta retries, and the retries arrive while you’re still slow. The fix is to make the request path trivially cheap.

With Oban, the job table is a perfectly good inbox. Insert one job per POST (not one per message) and return:

defmodule MyAppWeb.WhatsAppWebhookController do
  use MyAppWeb, :controller

  alias MyApp.Workers.ProcessWhatsAppWebhook

  def verify(conn, %{"hub.mode" => "subscribe", "hub.verify_token" => token, "hub.challenge" => challenge}) do
    if Plug.Crypto.secure_compare(token, Application.fetch_env!(:my_app, :whatsapp_verify_token)) do
      send_resp(conn, 200, challenge)
    else
      send_resp(conn, 403, "")
    end
  end

  def create(conn, _params) do
    case Oban.insert(ProcessWhatsAppWebhook.new(%{"payload" => conn.body_params})) do
      {:ok, _job} -> send_resp(conn, 200, "")
      {:error, _reason} -> send_resp(conn, 500, "")
    end
  end
end

The 500 is deliberate: if you couldn’t persist the event, you want Meta to send it again. Anything that can fail after persisting is your problem to retry, not Meta’s.

The job then fans the payload out:

defmodule MyApp.Workers.ProcessWhatsAppWebhook do
  use Oban.Worker, queue: :whatsapp_webhooks, max_attempts: 10

  alias MyApp.WhatsApp.{Inbound, Statuses}

  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"payload" => payload}}) do
    for %{"changes" => changes} <- Map.get(payload, "entry", []),
        %{"value" => value} <- changes do
      phone_number_id = get_in(value, ["metadata", "phone_number_id"])
      Enum.each(Map.get(value, "messages", []), &Inbound.handle(phone_number_id, &1))
      Statuses.apply_batch(Map.get(value, "statuses", []))
    end

    :ok
  end
end

Deduplicate: webhooks are at-least-once

You will receive the same inbound message more than once: after a retry, after a timeout where you did process it but Meta didn’t see the 200, occasionally for no visible reason. Every inbound message has an id (the wamid.… string). Put a unique index on it and let the database arbitrate:

def handle(phone_number_id, %{"id" => wamid, "from" => wa_id, "timestamp" => ts} = message) do
  attrs = %{
    wamid: wamid,
    wa_id: wa_id,
    phone_number_id: phone_number_id,
    payload: message,
    sent_at: DateTime.from_unix!(String.to_integer(ts))
  }

  case Repo.insert(InboundMessage.changeset(attrs), on_conflict: :nothing, conflict_target: :wamid) do
    # With on_conflict: :nothing, a duplicate comes back with no primary key.
    {:ok, %InboundMessage{id: nil}} -> :duplicate
    {:ok, inbound} -> ContactServer.dispatch(inbound)
  end
end

One subtlety: if the insert succeeds and processing then crashes, a retried job sees a duplicate and skips it, and the message is never handled. Store a processed_at column, set it when the conversation logic has finished, and have a periodic job pick up rows that were stored but never processed. “Seen” and “handled” are different states.

If the messages table is partitioned, a unique index on the message id alone isn’t possible (unique indexes must include the partition key). A small dedup table keyed by wamid solves it; I cover this in the Postgres partitioning post.

Keep each conversation in order

A user who sends “hi”, “I need help” and a photo in quick succession expects the bot to see them in that order. Two concurrent workers processing two messages from the same user will eventually get it wrong, and a chatbot state machine processing two inputs at once is a race condition with a user interface.

In a single node, the simplest correct approach is one process per contact:

defmodule MyApp.Conversations.ContactServer do
  use GenServer, restart: :transient

  @idle_timeout :timer.minutes(5)

  def dispatch(%{wa_id: wa_id} = inbound) do
    pid =
      case DynamicSupervisor.start_child(MyApp.ContactSupervisor, {__MODULE__, wa_id}) do
        {:ok, pid} -> pid
        {:error, {:already_started, pid}} -> pid
      end

    GenServer.cast(pid, {:inbound, inbound})
  end

  def start_link(wa_id), do: GenServer.start_link(__MODULE__, wa_id, name: via(wa_id))

  defp via(wa_id), do: {:via, Registry, {MyApp.ContactRegistry, wa_id}}

  @impl true
  def init(wa_id), do: {:ok, %{wa_id: wa_id}, @idle_timeout}

  @impl true
  def handle_cast({:inbound, inbound}, state) do
    :ok = MyApp.Bot.handle_inbound(state.wa_id, inbound)
    MyApp.WhatsApp.Inbound.mark_processed(inbound)
    {:noreply, state, @idle_timeout}
  end

  @impl true
  def handle_info(:timeout, state), do: {:stop, :normal, state}
end

Messages for one contact are handled strictly one after another; different contacts run in parallel; idle processes go away after a few minutes. Across a cluster you need every event for a contact to reach the same place: hash the contact id to a partition owned by one node, use a distributed registry, or use a job queue partitioned by contact with a concurrency of one per partition (Oban Pro supports partitioned limits for this kind of thing). Pick one and make it explicit, because “usually the same node” is not ordering.

Arrival order is not guaranteed either. When messages arrive within a second or two of each other, holding them briefly and sorting by the payload timestamp helps, but that timestamp has one-second resolution, so ties fall back to arrival order.

Status webhooks arrive out of order

For messages you send, you’ll receive statuses such as sent, delivered, read and failed, and not necessarily in that order: read can arrive before delivered. Treat status as monotonic. Rank the states and only apply an update if it moves the message forward, and apply them in batches, since statuses arrive at a multiple of your send rate. I go through the SQL for this in the bulk sending post.

If you set biz_opaque_callback_data when sending, it comes back on the status webhooks for that message, which is handy for correlating statuses with your own records without an extra lookup.

The 24-hour customer service window

When a user messages you, a 24-hour customer service window opens, during which you can reply with free-form messages. Outside the window you can only send approved template messages, and a free-form send fails (error 131047). When the user replies, a new window opens.

Don’t discover this from errors. Store the timestamp of each contact’s last inbound message (it’s already in your inbound table) and check it before sending. Leave a margin: a reply your bot generates 23 hours and 59 minutes in may land after the window closes. Design flows so that anything that might go out later, such as reminders, follow-ups or a human agent answering the next morning, has a template ready.

Media: download promptly

Inbound media messages don’t contain the file. They contain a media id (plus MIME type and a hash). Getting the bytes is two authenticated requests: fetch the media metadata by id, which returns a short-lived URL (Meta documents it as valid for five minutes), then download from that URL with the same bearer token.

def download_media(media_id) do
  version = Application.fetch_env!(:my_app, :graph_api_version)
  auth = [auth: {:bearer, Application.fetch_env!(:my_app, :whatsapp_token)}]

  with {:ok, %{status: 200, body: %{"url" => url, "mime_type" => mime}}} <-
         Req.get("https://graph.facebook.com/#{version}/#{media_id}", auth),
       {:ok, %{status: 200, body: bytes}} <- Req.get(url, auth ++ [decode_body: false]) do
    {:ok, bytes, mime}
  end
end

Do this in its own background job, soon after the webhook arrives, and copy the file into your own storage. Don’t store the URL, it will have expired by the time anyone clicks it. Don’t do it in the request path either: a large video download is exactly the slow step that makes webhooks time out. Size limits and virus scanning belong in the same job.

What breaks at scale

  • Reply spikes after campaigns. Send a message to a million people and a slice of them reply within minutes. Your webhook traffic is shaped by your outbound traffic. Load test the inbound path at the peak your biggest campaign will create.
  • Slow handlers turning into retry storms. Any synchronous work in the request path eventually gets slow, and slow becomes timeouts, retries and duplicates. Keep the request path to verify, persist, respond.
  • Database connection pool exhaustion. Webhook requests, the job queue and the conversation workers all compete for the same pool. Size it and watch queue time in the pool, not just query time.
  • Hot contacts. A single number stuck in a loop (often another bot) can flood one contact process. Add per-contact rate limits and loop detection.
  • Deploys. Rolling restarts kill in-flight processing. That’s fine if everything is persisted before the 200 and processing is idempotent; it’s data loss if it isn’t.
  • Logging whole payloads. They contain phone numbers and message content. In health services in particular, treat them as sensitive data.

Building on the WhatsApp Business Platform

The webhook receiver is the front door of any WhatsApp integration, and the most important rule for it is also the simplest: verify, persist, respond, and do everything else afterwards. If you’re building on the WhatsApp Cloud API, or your current integration struggles under load, I help teams build on the WhatsApp Business Platform.

Keep reading