Application Business Logic Audit: Payments, Refunds and Retries

Payment defects rarely fit inside one request. The dangerous cases appear between a shop database, a worker, a payment provider, and a webhook receiver: the provider may have accepted a charge while the application timed out; a webhook may be delivered twice or after a newer event; a refund may look successful and later fail; a database retry may repeat code that already caused an external effect.

This article uses Northstar Shop, a fictional online shop with monthly subscriptions, as a recurring example. Northstar creates orders locally, sends payment and refund requests to a provider, grants subscription access after payment, and keeps an internal financial ledger. The scenario is illustrative; it is not a customer case and does not claim measured financial results.

The relevant audit question is not “does the endpoint have an idempotency key?” It is: can the agreed retry, delay, duplicate, reordering and partial-failure scenarios charge a customer twice, lose a refund, or grant the wrong access? The answer is tested against financial rules that must remain true across the application and provider. This narrower question complements a general guide to business logic security testing and the authorization and evidence model in a REST API security assessment.

Start with financial invariants, not happy-path statuses

Northstar has an orders table, a payment_attempts table, a subscription service, and provider objects. A single label such as paid hides too much. The application needs to distinguish a locally requested operation, a provider-accepted operation, a provider-confirmed outcome, a locally committed entitlement, and a reconciled financial record.

For the fictional scenario, the audit begins with explicit invariants:

• one checkout intent can produce at most one economically effective charge;
• subscription access is granted only for the intended customer, plan, amount, currency, and billing period after authoritative confirmation;
• a duplicate delivery can repeat processing work but cannot repeat a ledger entry, shipment, credit, or entitlement;
• cumulative successful refunds cannot exceed the captured amount after accounting for prior partial refunds, disputes, reversals, and provider rules;
• a pending, requires_action, failed, or later-reversed refund is not presented as final money returned;
• every provider-side payment or refund can be correlated to one local business operation, while unmatched and mismatched records enter reconciliation rather than being silently forced into a status;
• acknowledging a webhook means its authentic payload is durably retained, not that every downstream business action already succeeded.

The last distinction is central. A transport acknowledgement, a provider API response, a local database commit, fulfillment, and settlement are different facts. Collapsing them into one success flag makes failures invisible.

Idempotency is scoped by the provider contract

“Use idempotency” is incomplete advice. The auditor must pin the provider, endpoint, API generation, account scope, region, retention window, parameter comparison rules, concurrency behavior, and response semantics. There is no universal exactly-once or idempotency guarantee across providers.

Stripe separates API v1 from v2. V1 binds parameters and can replay a saved 500; its finite key cache means later reuse can start a new request. V2 does not repeat successful work but can continue failed or partial work and return an updated response. An audit must pin the version.

Adyen scopes keys to a company account, retains them for 7–14 days, and does not deduplicate across regional endpoints. It identifies retryable transient errors and expects the same key on a retry. Northstar therefore pauses and reconciles when that protection is unavailable instead of weakening its duplication control.

An application-level operation ID must therefore outlive provider cache windows. Northstar generates one immutable refund-operation ID, stores it before the first remote call, binds the provider idempotency key and request parameters to that operation, and reuses them for legitimate retries. A support agent pressing “retry” must resume the same operation when the business intent is unchanged; creating a new key is a new economic command and requires a deliberate decision.

The test also checks collision and mutation controls. Reusing one key for a different amount, currency, customer, payment object, or refund reason should fail locally before a provider call. Two workers claiming the same pending operation should not produce two keys. A retry after the provider retention window should query and reconcile authoritative provider state before deciding whether a fresh command is safe.

Treat payment and application state as two interacting machines

Northstar’s local order might move through payment_requested, provider_unknown, provider_confirmed, entitlement_committed, and reconciled. The provider has its own states and may change them asynchronously. Neither state machine is a copy of the other, and a local transition must state what evidence authorizes it.

Consider a timeout after Northstar sends a payment request. Three realities are possible: the provider never began; the provider completed but the response was lost; or the provider is still processing. Setting the order to failed invents certainty. Sending a new request with a new key risks a second charge. The safe local state is unknown or pending, linked to the original operation and key. A worker retries according to the pinned provider contract, retrieves the provider object when possible, and lets a verified webhook or reconciliation close the uncertainty.

Database transactions protect local writes, not remote effects. PostgreSQL documents that serialization failures require retrying the complete database transaction, including the logic that chose the statements and values. If that transaction body directly calls the provider, retrying the SQL transaction can repeat an external payment that PostgreSQL cannot roll back. The provider call belongs outside an automatically retried database transaction, behind a durable command record and application-level idempotency boundary.

The transactional outbox pattern solves the local dual-write problem: Northstar commits the business transition and an outbox command in the same transaction; a worker later sends the command. It does not create global exactly-once execution. Relays can publish duplicates, so the receiver and downstream consumer still need durable deduplication and idempotent effects. The same principle applies in reverse with an inbox table for webhooks.

1. Record intentCommit the order, immutable operation ID, expected amount and currency, and an outbox command in one local transaction.
2. Call providerA worker sends the pinned request with the stored idempotency key and records the provider request and object identifiers.
3. Ingest outcomeVerify each webhook, durably insert its delivery identity and payload, then acknowledge before business processing.
4. Apply transitionA worker commits the ledger transition and an entitlement command; the entitlement consumer applies that command idempotently.
5. ReconcileCompare provider and local records, route mismatches to an exception queue, and retain the resolution trail.

Webhooks require durable intake, deduplication, and valid current transitions

Providers retry webhooks and may deliver them out of order. Stripe advises tracking event IDs for duplicate deliveries. Adyen notes that messages for the same eventCode and pspReference can carry different dates or details. Northstar must distinguish a repeated delivery from a genuine newer update for the same provider object: it retains and processes the update while preventing the underlying economic effect from being applied twice. One generic payload_hash rule is insufficient.

The receiver first verifies authenticity with the provider’s documented mechanism. It then inserts an inbox row under a unique provider-specific delivery identity, including the raw payload needed for processing, correlation values, receipt time, and verification outcome. Only after that insert commits does it return a successful acknowledgement. Adyen’s sequence expressly puts verification and durable storage before a 2xx response, and AWS asynchronous guidance says an acknowledgement should wait until durable persistence.

This design avoids two opposite losses. If Northstar acknowledges before durable storage and crashes, the provider may believe delivery succeeded while the application has no event. If Northstar performs all business logic before acknowledging, a slow entitlement service can trigger provider retries and duplicate work. Durable intake separates delivery from processing.

Deduplication has at least three levels:

• delivery identity and update version prevent reprocessing the same delivery without discarding a genuine later update for the same business object;
• business operation identity prevents two different events or API responses from applying the same refund, renewal, credit, or fulfillment twice;
• ledger uniqueness prevents the same economic fact from posting twice even if an upstream guard fails.

Out-of-order handling must be based on valid transitions and authoritative state, not arrival time alone. A stale intermediate event must not overwrite newer evidence, yet a later failure or returned bank transfer can legitimately change what once looked successful. If a late payment-success event arrives after checkout expiry, expiry does not make an authentic financial event disappear; the application verifies current provider state, then fulfills, refunds, or sends the item to review according to policy.

Refunds are commands with non-final intermediate states

A refund button is not a ledger reversal. It starts a separate operation with its own identity, authorization, limits, provider object, asynchronous statuses, and exception handling. Provider documentation describes pending-to-success, apparent-success-to-failure, and for some bank-detail flows a later requires_action. Northstar must not promise permanent completion from the initial response.

Partial refunds make the invariant multi-row and concurrent. Two support agents can each see all 100 units as available from a 100-unit capture and submit 60 at the same time. A provider may enforce a ceiling, but Northstar still needs a local reservation or atomic calculation to give deterministic operator feedback and avoid relying on a vendor rejection as its only control. The calculation must include successful refunds, reserved pending refunds, relevant failed or canceled reservations, disputes, reversals, fees when the business rule includes them, and the original currency’s minor-unit rules.

Local command stages such as requested and submitted should be tracked separately from the provider's refund status. Values such as pending, requires_action, succeeded, failed and canceled apply where the chosen API defines them. Mapping these into a smaller internal vocabulary is acceptable only when the mapping is explicit and retains the distinctions needed for customer communication, available-balance decisions, and reconciliation.

Subscription refunds add another invariant: money and entitlement need separate, deliberate transitions. Refunding a renewal does not automatically define whether access ends immediately, at period end, or after an operator review. Conversely, revoking access must not be treated as proof that money returned. The audit asks product and finance owners to approve the coupling policy and then tests it under retries and delayed events.

Partial failures need a designed recovery path

For each cross-boundary step, the test matrix asks what happens if the process stops immediately before and after the step. Useful fault points include after the local operation commit but before the worker sends; after the provider accepts but before Northstar records the response; after durable webhook intake but before processing; after the ledger entry but before entitlement; and after entitlement but before the outbox message is marked complete.

Every fault must lead to one of three controlled outcomes: retry the same idempotent operation, derive current state from authoritative provider evidence, or place the case in an exception queue for bounded human review. “Log and continue” is not a recovery strategy. Neither is an infinite retry loop that creates permanent pending orders with no owner.

Reconciliation closes gaps that online processing cannot eliminate. Northstar periodically compares local operation IDs, provider object IDs, amounts, currencies, statuses, and economic totals. The useful output is an exception list: provider charge without local order, local paid order without provider confirmation, refund total mismatch, disagreement about current confirmed state, missing webhook intake, or entitlement inconsistent with confirmed payment.

Reconciliation must be safe to rerun. Its corrective actions use the same operation identities and uniqueness controls as online processing. It records why a mismatch was resolved automatically, escalated, or accepted. Thresholds and time windows reflect the payment methods in scope; a delayed bank method and an immediate card flow should not share an arbitrary “five-minute” rule.

Build a safe test matrix around failures and interleavings

Testing should use provider sandboxes, synthetic users, dedicated orders, and non-production entitlements unless a separately authorized production case is essential. The matrix is tied to the exact provider mode and API version. Provider test tools can simulate useful states, but they do not prove how every banking rail behaves in production.

Concurrency bursts are agreed in advance and bounded. Tests do not attempt to load the provider. Logs and evidence exclude secrets and unnecessary payment data. A useful API pentest can include these application-owned integration paths when the rules of engagement name the sandbox, accounts, callback endpoints, allowed fault injection, and prohibited real-money effects.

Accept the report on state evidence, not screenshots

A payment-logic finding should name the violated invariant, starting state, exact request and event sequence, provider contract and version, local rows or ledger entries affected, and final economic effect. It should distinguish observed evidence from a hypothetical consequence. A screenshot of two 200 responses is insufficient if it does not show whether one or two provider objects, ledger postings, refunds, or entitlements resulted.

Before acceptance, the report should contain:

• a boundary diagram and state mapping for each in-scope payment and refund flow;
• the pinned idempotency semantics for every provider endpoint tested, including scope, retention, region, mutation, concurrency, and error behavior;
• an invariant-to-test matrix with normal, retry, duplicate, out-of-order, concurrent, timeout, crash, and reconciliation cases;
• correlation evidence joining the local operation, provider request and object, webhook delivery, ledger entry, and entitlement action;
• reproducible findings with sanitized timestamps and identifiers, plus limitations where the sandbox cannot reproduce a live behavior;
• retest evidence showing the original sequence and nearby variants, including failure injection at the commit boundaries;
• a reconciliation result with synthetic exceptions and proof that rerunning correction does not duplicate an effect.

“No duplicate charge observed” is weaker than “the same stored operation was retried at these boundaries; the provider showed one object, the local ledger had one unique posting, and reconciliation returned no unexplained duplicate for the test population.” Even that evidence is scoped and time-bound. It does not guarantee future releases, every provider rail, or universal exactly-once behavior.

The buyer should also reject a report that treats a provider’s controls as a substitute for application controls. Provider refund ceilings do not prove Northstar communicates status correctly, maps the right order, preserves tenant separation, or reconciles partial failures. Local database isolation does not make a remote effect transactional. A queue marked FIFO does not prove every downstream side effect happens once.

Scope the assessment so the conclusion is useful

The scope should identify checkout and renewal variants, refund and cancellation paths, currencies, payment methods, provider accounts and regions, API versions, webhook formats, subscription entitlement rules, and ledger systems. It should include test identities for at least two customers and operator roles, provider sandbox access, event replay facilities, logs with correlation IDs, and a way to inject bounded failures.

Agree exclusions plainly: provider infrastructure is not being penetration-tested unless separately authorized; production load testing and real-money movement are excluded; sandbox behavior is evidence about the integration under those conditions, not a promise about every live network. The pentest scope planner helps turn these boundaries into a testable brief, and the sample pentest report shows the level of evidence that should make a finding reviewable.

Pentest.red can assess the application-owned payment and refund logic as part of an authorized API or web application engagement. The practical next step is to provide the provider/API versions, the local state model, the critical financial invariants, and the allowed sandbox fault scenarios. That lets the assessment test the boundary where money and application state can diverge, without promising a universal guarantee or a predetermined financial outcome.