Form submissions usually get lost because the handler tries to send the email during the web request, and treats a failure as the end of the story. If the mail provider has a bad minute or the server restarts, the lead is gone, and often nobody knows. The outbox pattern fixes this by saving the submission and the job to send it in one database transaction.
Where a lead can go missing
Here's the handler most contact forms start with.
app.post("/contact", async (req, res) => {
await sendEmail(req.body);
res.json({ success: true });
});It's short and it works in testing. It also has several ways to lose a lead.
The mail provider has a problem. sendEmail throws, and the visitor sees an error, if they're lucky. Most won't fill the form in again. And the message they typed exists nowhere.
The process dies mid-request. A deploy or a crash during sendEmail ends the request. Maybe the email went, and maybe it didn't. Nothing records which.
The email is accepted, then bounces. The provider said OK, so the handler said success. A full mailbox or a blocked sender shows up later, somewhere nobody is looking.
The webhook endpoint is down. Your CRM is being deployed when the form posts. There's no second attempt.
Saving to a database first looks like the fix.
const row = await db.insert(submissions).values(req.body);
await sendEmail(req.body);
res.json({ success: true });Better, because the data now exists. But a crash between those two lines leaves a saved submission that nothing will ever send. It sits in a table until someone happens to look. That's the dual-write problem: you've got two systems to update, and no way to update both atomically.
The outbox pattern
The fix is to stop doing the second write during the request. Instead, write a record of the work in the same transaction as the data.
BEGIN;
INSERT INTO submissions (id, form_id, data)
VALUES ($1, $2, $3);
INSERT INTO deliveries (submission_id, channel, target, status)
VALUES ($1, 'email', $4, 'pending'),
($1, 'webhook', $5, 'pending');
COMMIT;Either both rows exist or neither does. Once that commit returns, you can tell the visitor it worked, and that will be true even if the server dies on the next line. No email has been sent yet, but the intent to send one is now durable.
A background worker does the sending.
UPDATE deliveries
SET locked_until = now() + interval '2 minutes', attempts = attempts + 1
WHERE id IN (
SELECT id FROM deliveries
WHERE status = 'pending' AND next_attempt_at <= now()
AND (locked_until IS NULL OR locked_until < now())
ORDER BY next_attempt_at
LIMIT 25
FOR UPDATE SKIP LOCKED
)
RETURNING *;FOR UPDATE SKIP LOCKED lets any number of workers run at once without claiming the same row. locked_until means that if a worker dies holding a job, the job becomes available again a couple of minutes later.
Retries with backoff
A failed attempt goes back to pending with a later next_attempt_at, and the wait grows each time. Most outages are short, so early retries catch them. Later ones cover the long ones without hammering a service that's struggling.
Not every failure deserves a retry. A 410 Gone from a webhook, or a Telegram bot that's been removed from a chat, won't fix itself. Mark those as failed straight away and show the reason.
Making failure visible
Storing the state gives you something most form handlers lack: you can see what happened. Each delivery has a status, an attempt count and its last error. "Did that lead reach us?" becomes a lookup rather than a guess, and after you've fixed the cause, you can send the failed one again.
One side effect to design for
An outbox gives you at-least-once delivery. If a worker sends a webhook and dies before recording success, that webhook goes out again. So receivers need to handle duplicates. Send a stable delivery ID in a header, and have the receiver ignore IDs it has already seen.
Also: failing over the email itself
Retries help when a provider has a brief problem. A second provider helps when the problem lasts longer. Send through your main provider, and if it fails with a temporary error, try the next one. Permanent errors, such as a rejected address, shouldn't fail over, because the second provider would reject it too.
How FormThunder does this
FormThunder uses this design. A submission and its delivery jobs are written to Postgres in one transaction before we return success. Email, webhook and chat deliveries retry with exponential backoff for up to 24 hours across 10 attempts, and email fails over from Amazon SES to Resend. Every attempt is visible and failed ones can be resent. There's more on the reliability page.
If you're building your own handler, you don't need our product to use any of this. It takes two tables and a worker loop. The rest of the work is in the small details: the locks, the backoff and deciding which errors are permanent.