To prevent an integration from creating duplicate invoices, orders or payments, every business operation needs a stable identity that is retained across all retry attempts. The receiving system must register this identity atomically, block repeated execution and return the existing result when a request is repeated. An idempotency key in an API header alone is not enough: protection must cover the database, concurrent requests, message queues and external services. In practice, a robust solution combines an idempotency key, a unique database constraint, a stored operation status and regular reconciliation of operations with uncertain outcomes.
What idempotence means in an integration
An idempotent operation is one in which repeating the same request does not change the intended result after the first successful execution. For example, if a request to create an order is sent three times because of a network interruption, the system should still contain only one order. Retry attempts can receive the identifier of the previously created order instead of creating new records.
HTTP method semantics alone do not guarantee this outcome. RFC 9110 defines certain methods as idempotent, but the business side effects of a particular system must still be designed correctly. Conversely, POST is not idempotent by default, but a specific POST endpoint can be made safe to retry by using an idempotency key and server-side controls.
It is important to distinguish a technical request from a business operation. If a user intentionally creates two identical orders, these are two operations. If a client repeats one request after a timeout, it is one operation with multiple delivery attempts. The idempotency mechanism must be able to distinguish between these cases.
Why duplicates occur
A duplicate does not usually arise because a system deliberately executes a command twice. A more common cause is an uncertain outcome. A client sends a request, the server creates an invoice, but the response is lost. The client sees a timeout and retries without knowing that the first operation has already been completed.
A similar situation can be caused by:
- an automatic retry by an HTTP client or proxy server;
- redelivery of a message after a consumer failure;
- two concurrent workers or background tasks;
- a user clicking repeatedly;
- repeated receipt of a webhook event;
- a process resuming after a server failure;
- manual execution of a stalled task without checking its previous state.
Preventing retries is therefore not a safe strategy. In a distributed system, retries are often necessary to overcome temporary failures. The goal is to make each retry controlled so that it does not change the business outcome.
Implementing idempotence in six layers
1. Assign an identity when the business operation begins
The idempotency key should be created in the system where the intention to perform the operation originates. It can be a randomly generated value or a stable business identifier if its uniqueness is clearly defined. The essential requirement is that all retries of the same operation use the same key.
If a new key is generated for every retry, the receiving system sees new requests and cannot recognise the duplicate. The scope of the key must also be defined precisely. It is safer to evaluate it together with the client or tenant identifier and the operation type, for example, uzņēmums + izveidot-rēķinu + atslēga. This prevents one client’s key from accidentally blocking another client’s operation.
2. Bind the key to immutable request content
The server should store not only the key but also a fingerprint of the relevant input data or fields that can be compared in canonical form. If the same key is received again with a different amount, currency, order contents or recipient, the request must not silently be treated as a successful retry.
In this situation, the server should return a conflict and record diagnostic information. Otherwise, the client might believe that the new version was executed even though the server returned the result of the first operation. Data normalisation must also be defined because semantically identical JSON documents can differ in field order or insignificant formatting details.
3. Reserve the operation atomically in the database
Before executing the business side effect, the server attempts to create an idempotency record with a unique key. A unique database constraint should be the main protection against a race condition, rather than relying only on a preliminary SELECT query.
Two concurrent processes can both check that the record does not exist and then both continue. A unique index or constraint allows only one of them to obtain the right to begin the operation. PostgreSQL INSERT ... ON CONFLICT is one technical tool for making this reservation, but after a conflict the existing record must still be read, its content compared and its status evaluated.
4. Store the status and return the result to a client that retries
An idempotency record should contain at least the operation key, scope, input fingerprint, status, creation time and a reference to the outcome. Typical statuses may include processing, succeeded, failed and unknown, but their meaning must be defined unambiguously within the particular system.
If the operation has already been completed, a repeated request should receive the stored result or a reference to the created object. If it is still being processed, the server can indicate that the result is not ready or provide a status-check endpoint. This is safer than starting a second execution in parallel.
Not every error should be stored as final. A validation error before the reservation differs from an error after an external payment has been processed. The status model must show whether the operation may be retried or whether its actual outcome must first be established.
5. Protect the business object itself
The idempotency table is not the only layer of protection. The business table also needs a uniqueness rule that reflects the meaning of the process. For an invoice, this could be a combination of the source order and invoice type; for an order, it could be the external system’s order identifier combined with the source. The precise model depends on whether one order may have several partial invoices, corrections or separate deliveries.
This constraint also provides protection if the API idempotency layer is bypassed because of an error. However, an overly broad uniqueness rule can block a legitimate repeat purchase. The unique business key should therefore be based on the process rules, not on the assumption that the same amount and client necessarily indicate a duplicate.
6. Pass the same identity to external services
If the local system calls a payment, invoicing or delivery service, idempotence must continue across that boundary. If the service supports idempotency keys, the same key should be used in every retry. The Stripe API documentation is one publicly documented example of this approach for POST requests.
If the external service does not support idempotence, its business identifiers, status checks and a reconciliation process must be used. After a timeout, the system must not automatically assume that the operation did not take place. It should first check the status in the external system. If the outcome cannot be determined, the local operation should remain in an uncertain state rather than immediately creating another payment or invoice.
Practical example of creating an invoice
Suppose a CRM sends an ERP a command to create an invoice for order A-1048. Before the first attempt, the CRM creates an operation key and stores it with the task. The ERP receives the request and, within one database transaction, reserves the idempotency record and creates the invoice or a safe instruction for its creation.
If the response is lost, the CRM repeats the request with the same key. The ERP finds the completed record and returns the identifier of the invoice that has already been created. If two requests arrive simultaneously, the unique constraint allows only one of them to begin execution.
An additional safeguard is a unique link between the source order and the specific role of the invoice. It must be modelled precisely enough not to interfere with advance, final or correction documents if the process allows them. Idempotence defines the behaviour of a repeated technical command, but it does not itself determine the company’s invoice numbering or correction procedures.
Message queues, outbox and inbox
In a message queue, a consumer can complete a business operation but fail before acknowledging receipt. The broker then delivers the message again. RabbitMQ documentation on consumer acknowledgements explains the mechanism that requires an application to be prepared for redelivery.
The outbox approach makes it possible to store both the business change and the event to be sent within one local database transaction. A separate process publishes the event and may retry. The recipient’s inbox or deduplication table records the identifiers of processed messages and prevents the same operation from being executed again.
The outbox does not resolve everything automatically. Publishing and delivery can still be repeated, but the retry becomes safe. In addition, a single event identifier does not always protect the business operation: two different events may attempt to create the same invoice. Both message deduplication and business-level uniqueness are therefore required.
What to test before implementation
Idempotence cannot be confirmed with a single successful API test. Tests should reproduce uncertainty and concurrent execution:
- Send one request several times in succession with the same key and verify that only one object is created.
- Send several simultaneous requests with the same key and test the database’s unique constraint.
- Repeat the key with a changed amount or recipient and expect an explicit conflict.
- Interrupt the connection after server-side execution but before receiving the response, then repeat the request.
- Crash the consumer after the database transaction but before acknowledging the message.
- Simulate an external service timeout and verify that the system first determines the status of the previous attempt.
- Test the operation after the retention period for the deduplication record has expired.
- Run the manual fallback process and ensure that it uses the same uniqueness rules.
The retention period should not be chosen solely on the basis of database costs. It must cover the possible period of retries, delayed delivery and reconciliation. If a record is deleted too early, an old retry appears to be a new operation again. At the same time, longer retention increases data volume and may require sensitive fields to be minimised.
Monitoring and reconciliation
It is useful to monitor successful repeated requests, conflicts between keys and content, processing records that remain stuck for a long time, operations with an unknown status and discrepancies between the local and external systems as separate metrics. These indicators help distinguish a normal retry from an integration defect or incorrect client behaviour.
Processes involving money and documents require a reconciliation task that compares records in both systems using a stable external reference. Idempotence reduces the likelihood of duplicates, but it cannot prove that a remote operation took place if neither the external system’s response nor a status query is available.
Limitations and risks
Idempotence does not make an incorrect command correct. If the source system mistakenly creates two separate operations with different keys, the technical mechanism may execute both. Business validation, approval rules and an appropriate unique key are needed to prevent this problem.
An idempotency key is not a substitute for authentication and authorisation. The server must still verify that the requester is entitled to perform the operation and access the previously stored result. The key must not be used as secret proof of access.
Care must also be taken to avoid a scope that is too narrow or too broad. One key for an entire client would block legitimate operations, while a key held only in one process’s memory would provide no protection after a server restart. The deduplication state must be held in persistent storage accessible to every worker handling the relevant endpoint.
The most difficult cases are side effects outside the local transaction: an email may have been sent, a payment accepted or a document created immediately before a failure. These cases require idempotence in the external service, a status check or reconciliation. A promise of absolute exactly-once execution is not justified when there is no shared atomic boundary across all participating systems.
Recommended internal links
Relevant material for the reader’s next steps includes documenting process boundaries in biznesa-procesa-kartesana-pirms-automatizacijas, choosing the integration type in api-integracija-vai-vienkarsa-datu-apmaina, and designing a manual fallback process in automatizacijas-kludu-apstrade-manualais-rezerves-process.
Conclusion
Safe idempotence begins not by prohibiting retries, but by recognising a single business operation throughout the integration chain. A stable key must reach the database, queue and external service, while unique constraints and stored status must stop concurrent execution. If the external outcome cannot be determined, the system should acknowledge this and direct the operation to a status check or reconciliation instead of blindly creating the next action.
