An AI code-review bot on Lambda and LiteLLM

Pull requests waiting for a reviewer were the team's bottleneck, and reviewers spent a good part of their time on things a machine would catch. I built a bot that reviews every PR as soon as it opens: a Bitbucket webhook calls a Lambda, the Lambda sends the diff to a model through a LiteLLM proxy, and the result comes back as comments on the PR, with an approval or a change request.

Wiring that up is easy. The work was in the details below.

1. Make sure the webhook is yours

Bitbucket signs the body with HMAC-SHA256 in the X-Hub-Signature header. The comparison uses timingSafeEqual, so the signature doesn't leak through response timing:

function isValidSignature(rawBody, signature) {
  const secret = process.env.WEBHOOK_SECRET || "";
  if (!signature || !secret) return false;
  const expected = "sha256=" + crypto.createHmac("sha256", secret).update(rawBody).digest("hex");
  const a = Buffer.from(expected);
  const b = Buffer.from(String(signature));
  return a.length === b.length && crypto.timingSafeEqual(a, b);
}

Watch the body: behind API Gateway it may arrive base64-encoded (isBase64Encoded). The signature covers the original text, so decode before validating.

The handler only reacts to pullrequest:created and pullrequest:updated, and skips PRs that aren't open or whose title starts with WIP: or draft.

2. Don't review the same commit twice

Bitbucket may deliver the same event more than once, and every push fires an updated. The dedup key is the source commit SHA, not a hash of the diff: a new commit always has a new SHA, but the diff can be identical (an amend that only changes the message, for example).

The lock is a conditional PutItem in DynamoDB:

await ddb.send(new PutCommand({
  TableName: table,
  Item: { pk, commitHash, status: "running", startedAt: now.toISOString() },
  ConditionExpression:
    "attribute_not_exists(pk) OR commitHash <> :hash" +
    " OR (#s = :running AND startedAt < :cutoff)",
  ExpressionAttributeNames: { "#s": "status" },
  ExpressionAttributeValues: { ":hash": commitHash, ":running": "running", ":cutoff": fifteenMinutesAgo },
}));

If the condition fails, another webhook is already handling that commit. The last clause covers a Lambda that died halfway: a running lock older than 15 minutes can be taken over. And if the review fails, the lock is released so the next event can try again.

3. Don't spend a model on the obvious

Before calling any LLM, the bot looks for conflict markers (<<<<<<<, =======, >>>>>>>) in the diff. If it finds them, it requests changes right away, without calling a model.

4. Big diffs in batches

A huge diff in a single prompt blows the model's time budget. The diff is split per file and packed into batches of up to 20,000 characters, with at most 6 batches reviewed in parallel:

export function splitDiffByFile(diff) {
  return diff
    .split(/^(?=diff --git )/m)
    .filter((p) => p.trim())
    .map((text) => ({ file: text.match(/^diff --git a\/.+? b\/(.+)$/m)?.[1] ?? "?", text }));
}

A file larger than a batch goes alone and truncated, and whatever doesn't fit in the 6 batches shows up in the summary as not reviewed instead of disappearing. When merging, one batch requesting changes is enough for the whole PR to get a change request.

5. Two reviews at once

The model with extended thinking finds more bugs: on a sample of real PRs it caught 3 out of 3, against 1 out of 4 for the fast model. But it sometimes runs out of time. So both versions are requested together, and the fast one (around 25 seconds) is only used if the other fails:

export async function reviewWithFallback(review) {
  const fast = review(false);
  fast.catch(() => {}); // only awaited if the other one fails
  try {
    return { ...(await review(true)), fast: false };
  } catch {
    return { ...(await fast), fast: true };
  }
}

The fast.catch(() => {}) is there so the fast promise doesn't become an unhandled rejection when nobody is waiting for it.

6. The timeout that wasn't mine

The first version used fetch, and long reviews died at 300 seconds. The reason: Node's fetch (undici) gives up if the response headers don't arrive within 300 s, and a model reading a 60,000-character diff sometimes takes longer. I switched to node:https, which has no such limit, and the only timeout became mine: 420 s, below the Lambda's 900 s.

Retries: just one, and only for 429, 502, 503 and 504 that happen within the first 60 seconds. A slow failure gets no second chance, because there'd be no Lambda time left for it.

And LiteLLM?

The Lambda doesn't know which model is on the other side. It calls /chat/completions on a LiteLLM proxy with one of the proxy's model names, asking for response_format: { type: "json_object" }. Switching models, providers or versions is proxy configuration, with no Lambda deploy.