Why Retries Break Your Java APIs (And How Idempotency Fixes It)

A practical guide to idempotency keys, IETF standards, and safe retries with Quarkus

Retries are a fundamental property of distributed systems. HTTP clients retry on timeouts, SDKs retry on transient failures, load balancers replay requests, and message brokers redeliver messages. None of these components know whether a request already caused a side effect.

Without idempotency, retries silently corrupt data.

With idempotency keys:

This is not theoretical. A €100 payment processed twice creates refunds, dispute fees, and immediate customer-facing damage. Users see the problem in their banking app within seconds.

Transactions, ACID databases, and HTTP status codes do not solve this. The failure happens above the database and below the business layer. Idempotency keys exist precisely to close this gap.

The IETF foundation

The concept is grounded in work by the Internet Engineering Task Force and builds on established HTTP semantics.

Relevant specifications:

• RFC 9110 (HTTP Semantics)
  Defines which HTTP methods are idempotent by definition (GET, PUT, DELETE) and which are not (POST).

• IETF draft: draft-ietf-httpapi-idempotency-key-header
  Defines the Idempotency-Key request header.
  Status: Working Group draft, not yet an RFC.

The contract is intentionally narrow:

• The client supplies a key representing one logical operation.

• The server uses that key to detect retries.

• The server replays the same result for the same key.

“Replaying the same result” typically means returning the same status code and response body. It does not freeze the entire system state. Other resources may change between retries. The guarantee is limited to the effect of that operation.

Why retries without idempotency corrupt data

Failures look clean. No exception. No rollback. Just duplicated state.

A less obvious example is a GET endpoint with side effects:

GET /download-report

If this triggers report generation and notification, retries can create duplicate files and duplicate emails. HTTP method semantics alone are insufficient. Any endpoint with side effects requires idempotency, regardless of verb.

Mapping the standard to Java and Quarkus

We will implement idempotency in a way that works for:

• REST APIs

• Concurrent requests

• Asynchronous and message-driven workflows

The stack:

• Java 21

• Quarkus

• PostgreSQL via Dev Services

• A database-backed idempotency store

How clients should generate idempotency keys

Key generation is the client’s responsibility.

Good options:

• UUIDv4

• UUIDv7 for time-ordered keys

• Scoped keys such as {tenant}:{user}:{operation}:{uuid}

Bad options:

• Timestamps only

• Sequential IDs

• Hashes of the request payload

Clients must persist the key across retries. A new key always represents a new logical operation.

Prerequisites

You need Java 21 and Maven and the Quarkus CLI to follow along. But you can also just grab the repository from my Github.

quarkus create app com.example:idempotency-demo \
  --extension=quarkus-rest-jackson,hibernate-orm-panache,jdbc-postgresql,quarkus-scheduler,micrometer
cd idempotency-demo

The idempotency data model

package com.example.idempotency;

import java.time.Instant;

import io.quarkus.hibernate.orm.panache.PanacheEntity;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Table;

@Entity
@Table(name = "idempotency_records")
public class IdempotencyRecord extends PanacheEntity {

    @Column(name = "idem_key", unique = true, nullable = false)
    public String key;

    @Column(name = "request_fingerprint", nullable = false)
    public String requestFingerprint;

    @Column(name = "status_code", nullable = false)
    public int statusCode;

    @Column(name = "response_body", columnDefinition = "TEXT")
    public String responseBody;

    @Column(name = "created_at", nullable = false)
    public Instant createdAt;

    @Column(name = "expires_at")
    public Instant expiresAt;
}

Idempotency service with correct error handling

Concurrency safety relies on the database, but error handling must be precise. Only constraint violations should be treated as “another request won the race.”

package com.example.idempotency;

import java.time.Instant;
import java.util.Optional;
import java.util.function.Supplier;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.persistence.PersistenceException;
import jakarta.transaction.Transactional;

@ApplicationScoped
public class IdempotencyService {

    @Transactional
    public Optional<IdempotencyRecord> find(String key) {
        return IdempotencyRecord.find("key", key).firstResultOptional();
    }

    @Transactional
    public IdempotencyRecord findOrCreate(
            String key,
            String requestFingerprint,
            Supplier<ProcessedResponse> processor) {

        try {
            ProcessedResponse response = processor.get();

            IdempotencyRecord record = new IdempotencyRecord();
            record.key = key;
            record.requestFingerprint = requestFingerprint;
            record.statusCode = response.status();
            record.responseBody = response.body();
            record.createdAt = Instant.now();
            record.expiresAt = Instant.now().plusSeconds(24 * 3600);

            record.persist();
            return record;

        } catch (PersistenceException e) {
            if (isConstraintViolation(e)) {
                return find(key).orElseThrow(() -> new IllegalStateException("Key exists but record not found"));
            }
            throw e;
        }
    }

    private boolean isConstraintViolation(PersistenceException e) {
        return e.getCause() instanceof org.hibernate.exception.ConstraintViolationException;
    }
}

Let’s define the record too:

package com.example.idempotency;

public record ProcessedResponse(int status, String body) {}

REST endpoint with fingerprint validation and metrics

Hashing utility

package com.example.idempotency;

import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.HexFormat;

public class Hashing {
    public static String sha256(String input) {
        try {
            MessageDigest digest = MessageDigest.getInstance("SHA-256");
            byte[] bytes = digest.digest(input.getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(bytes);
        } catch (Exception e) {
            throw new RuntimeException("SHA-256 unavailable", e);
        }
    }
}

REST resource

package com.example.api;

import com.example.idempotency.Hashing;
import com.example.idempotency.IdempotencyRecord;
import com.example.idempotency.IdempotencyService;
import com.example.idempotency.ProcessedResponse;

import io.micrometer.core.instrument.MeterRegistry;
import jakarta.inject.Inject;
import jakarta.ws.rs.BadRequestException;
import jakarta.ws.rs.Consumes;
import jakarta.ws.rs.HeaderParam;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.WebApplicationException;
import jakarta.ws.rs.core.Response;

@Path("/orders")
@Consumes("application/json")
@Produces("application/json")
public class OrderResource {

    @Inject
    IdempotencyService service;

    @Inject
    MeterRegistry registry;

    @POST
    public Response createOrder(
            String payload,
            @HeaderParam("Idempotency-Key") String key) {

        if (key == null || key.isBlank()) {
            throw new BadRequestException("Missing Idempotency-Key");
        }

        String fingerprint = Hashing.sha256(payload);

        IdempotencyRecord record = service.find(key)
                .map(existing -> {
                    registry.counter("idempotency.replays",
                            "endpoint", "orders").increment();
                    validateFingerprint(existing, fingerprint);
                    return existing;
                })
                .orElseGet(() -> service.findOrCreate(key, fingerprint, () -> process(payload)));

        return Response.status(record.statusCode)
                .entity(record.responseBody)
                .build();
    }

    private ProcessedResponse process(String payload) {
        return new ProcessedResponse(201, "{ \"status\": \"created\" }");
    }

    private void validateFingerprint(IdempotencyRecord existing, String current) {
        if (!existing.requestFingerprint.equals(current)) {
            throw new WebApplicationException(
                    "Idempotency key reused with different payload", 422);
        }
    }
}

Here is a manual, copy-paste-ready curl test section you can insert immediately after Section 8 (REST endpoint) in the article.

It is written in the same pragmatic, hands-on style and validates the behavior without any test framework.

Manual verification with curl

Before moving on to automated tests, it is useful to verify idempotency manually. This makes the behavior concrete and is often how issues are first diagnosed in production.

Start the application:

./mvnw quarkus:dev

First request (original call)

Send a POST request with a new idempotency key:

curl -X POST http://localhost:8080/orders \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000" \
  -d '{ "item": "book" }'

Expected result:

• HTTP 201 Created

• Response body:

{
  "status": "created"
}

At this point, the order is created and the idempotency record is stored.

Retry with the same key (safe retry)

Repeat the exact same request:

curl -X POST http://localhost:8080/orders \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000" \
  -d '{ "item": "book" }'

Expected result:

• HTTP 201 Created

• Identical response body

• No new database writes

This confirms the core guarantee:
multiple retries → one side effect.

Retry with same key but different payload (client error)

Now retry with the same key but a different payload:

curl -X POST http://localhost:8080/orders \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000" \
  -d '{ "item": "laptop" }'

Expected result:

• HTTP 422 Unprocessable Entity

• Payload:

{
  "error": "Idempotency key reused with different payload"
}

This protects you from subtle client bugs where retries are not actually retries.

New key = new operation

Finally, generate a new key:

curl -X POST http://localhost:8080/orders \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: 123e4567-e89b-12d3-a456-426614174999" \
  -d '{ "item": "book" }'

Expected result:

• HTTP 201 Created

• A new order is created

This reinforces the contract:
idempotency keys scope logical operations.

Verifying concurrency behavior

package com.example;

import io.quarkus.test.junit.QuarkusTest;
import io.restassured.RestAssured;
import org.junit.jupiter.api.Test;

import java.util.List;
import java.util.UUID;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.IntStream;

import static io.restassured.RestAssured.given;

@QuarkusTest
public class IdempotencyTest {

    @Test
    public void testConcurrentIdempotentRequests() throws Exception {
        String key = UUID.randomUUID().toString();
        ExecutorService pool = Executors.newFixedThreadPool(10);

        List<CompletableFuture<String>> futures = IntStream.range(0, 10)
                .mapToObj(i -> CompletableFuture.supplyAsync(() -> given()
                        .header("Idempotency-Key", key)
                        .body("{\"item\":\"book\"}")
                        .post("/orders")
                        .then()
                        .extract()
                        .body()
                        .asString(), pool))
                .toList();

        // Wait for all futures to complete and collect results
        CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();

        pool.shutdown();
    }
}

Messaging and async workflows

Idempotency applies unchanged to message processing (conceptual).

@ApplicationScoped
public class OrderConsumer {

    @Inject
    IdempotencyService service;

    @Incoming("orders")
    public CompletionStage<Void> process(Message<Order> msg) {

        String key = msg.getMetadata(KafkaMessageMetadata.class)
            .flatMap(m -> Optional.ofNullable(
                m.getHeaders().lastHeader("idempotency-key")))
            .map(h -> new String(h.value(), StandardCharsets.UTF_8))
            .orElseThrow(() ->
                new IllegalArgumentException("Missing idempotency-key header"));

        return service.find(key)
            .map(r -> CompletableFuture.completedFuture(null))
            .orElseGet(() ->
                CompletableFuture.runAsync(() -> handle(msg.getPayload()))
                    .thenRun(msg::ack));
    }
}

Production considerations

TTL cleanup

package com.example.idempotency;

import java.time.Instant;

import io.quarkus.scheduler.Scheduled;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.transaction.Transactional;

@ApplicationScoped
public class IdempotencyCleanupJob {

    /**
     * Runs once per hour.
     *
     * Removes idempotency records that are past their expiration time.
     * Uses a bulk delete for efficiency.
     */
    @Scheduled(every = "1h")
    @Transactional
    public void cleanupExpiredKeys() {
        long deleted = IdempotencyRecord.delete(
                "expiresAt IS NOT NULL AND expiresAt < ?1",
                Instant.now());

        if (deleted > 0) {
            // Keep logging lightweight; this is operational signal, not noise
            System.out.println(
                    "Idempotency cleanup removed " + deleted + " expired records");
        }
    }
}

• Do not delete aggressively
  Clients retry hours later. Use 24–72 hours unless you have strong guarantees.

• Index expires_at
  Already shown earlier. This keeps cleanup fast even at scale.

• Cleanup is best-effort
  Expired keys are harmless if they linger briefly.

In-flight requests

For in-flight detection, consider:

• An enum column: PROCESSING | COMPLETED | FAILED

• For high-concurrency environments, replace the standard find with a pessimistic lock to prevent two threads from seeing 'no record' simultaneously.

  SELECT ... FOR UPDATE SKIP LOCKED for application-level locking

This keeps retries predictable without blocking indefinitely. Might look like this:

Common pitfalls and troubleshooting

Expiring keys too early
Use generous TTLs, typically 24–72 hours.

Client generates a new key on retry
A new key is a new operation by definition.

Infrastructure drops headers
Ensure Idempotency-Key is preserved end-to-end.

Debugging replay failures

If a retry returns a different response:

1. Check created_at: was the key reused after expiration?

2. Compare request_fingerprint: did the payload change?

3. Inspect logs for concurrent processing

4. Verify the unique constraint exists:

SELECT * FROM pg_indexes WHERE tablename = 'idempotency_records';

Why this is not “just a Stripe thing”

Stripe popularized idempotency keys because payment systems make failures visible and expensive. The pattern itself is universal. Any system with retries and side effects needs it.

If your architecture uses at-least-once delivery anywhere, idempotency is not optional.

Conclusion and next steps

Idempotency keys are a small mechanism with disproportionate impact. They turn unreliable delivery into predictable behavior without coordination or guesswork. Anchored in IETF work and straightforward to implement in Java with Quarkus, they belong in every production API and message consumer.

Next steps:

• Implement this pattern on your highest-risk endpoint

• Apply the same logic to your messaging layer

• Enable replay metrics before deploying to production

The few lines of code you add today will save hours of debugging duplicate state tomorrow.

Read more about API design: