Quarkus Aesh: Your Application Just Grew a Terminal

Build one bounded operations console with tested commands, SSH key access, a protected browser terminal, health checks, and an audit trail.

Sep 30, 2026

I have spent a good part of my career telling people not to SSH into production servers.

And I still mean it.

Giving somebody a Linux shell inside a running application environment is usually a terrible operational interface. You get ps, kill, curl, filesystem access, environment variables, maybe database credentials, and eventually somebody runs the command they really should not have run. We moved away from this for good reasons. So when I saw that Quarkus can expose Aesh commands over SSH and WebSockets, my first reaction was not: great, let’s put a shell into the application.

It was: there are situations where I would really like to give an operator five commands and absolutely nothing else.

Imagine SwiftShip during an incident. The Berlin dispatch queue is backing up, but the application itself looks fine. The JVM is alive. HTTP responds. Kubernetes sees a healthy pod. The dashboard tells us that two shipments failed.

The problem sits one level above all of that.

Someone needs to see which shipments failed, inspect the proposed retry route, retry one of them, and perhaps stop Berlin from accepting new work while the carrier problem continues. These are operations on the running application. They are not part of the customer API, and I would not add them there just because operations needs them during an incident. This is where things usually become uncomfortable. I can give the operator access to the container and let them poke around. Too much access. I can give them database credentials and a SQL snippet. Now they can bypass the application rules completely. Or I can build a collection of protected admin endpoints and slowly create a second API with its own validation, authorization, documentation, and client scripts. None of those options feels particularly good.

What I actually want is much smaller: status, failures, retry, drain, and resume. And that is the idea behind this tutorial. The terminal is not a route to the operating system. It is an operations interface implemented inside the application boundary. Commands call the same CDI services as the rest of SwiftShip. They get validation, dry runs, confirmation, idempotency, and audit records. There is no rm, no SQL prompt, and no Runtime.exec() hiding behind it.

SSH then becomes just one way to reach that command vocabulary. The browser becomes another.

Quarkus currently provides three extensions for this. Aesh defines the commands. Aesh SSH and Aesh WebSocket expose them remotely. All three are preview extensions in Quarkus 3.39.1, so there are some important production caveats we will get to later.

For now, Berlin has two failed shipments. Let’s give the operator exactly enough power to deal with them. Berlin’s dispatch queue is backing up, but SwiftShip still responds. HTTP is healthy, the JVM is alive, and the dashboard reports two failed shipments.

An operator needs to inspect the failures, see the proposed retry route, retry one shipment safely, and perhaps stop new traffic. The public API deliberately has none of these operations. It serves customers and partners; it does not control a live hub.

A Linux shell or direct database access would solve the immediate problem by giving the operator far too much power. A protected admin endpoint looks narrower, but one endpoint soon becomes twelve sets of input rules and access checks. A bounded application terminal is the middle ground. SwiftShip exposes status, failures, retry, drain, and resume, but no general shell or SQL prompt. Dispatch fits this example because the targets are specific and the state changes deserve dry runs and confirmation.

We will connect that vocabulary to SwiftShip’s CDI service, add the necessary safety and evidence, then expose one command model through SSH and a browser.

Quarkus now provides three extensions for this use case. Aesh defines the commands. Aesh SSH and Aesh WebSocket add remote access. All three are preview extensions in Quarkus 3.39.1. Read the production section before you place this terminal on a real network.

What we are building

SwiftShip has three dispatch hubs. Berlin starts degraded with two failed shipments, while Madrid and Oslo are healthy. The data stays in memory so we can focus on the terminal boundary rather than persistence.

The finished application has three parts:

A small command set with live completion and input validation.
An SSH server and an xterm.js browser terminal with separate authentication rules.
Readiness checks, session events, and audit records for commands that change business state.

We move from orientation to the two failures, then preview a retry and decide whether to drain traffic. The terminal has no rm, curl, SQL prompt, or route to Runtime.exec(). Its authority stops at the application boundary.

Before you start

You need:

JDK 25 on PATH
The Quarkus CLI and the generated Maven wrapper
An OpenSSH client with ssh-keygen
About ☕️☕️☕️☕️ (This is a little more plumbing this time)

Create the application or start from the ready built project on my Github:

quarkus create app com.themainthread:quarkus-aesh-terminal \
  -P io.quarkus.platform:quarkus-bom:3.39.1 \
  --java=25 \
  --no-code \
  --extensions=aesh,aesh-ssh,aesh-websocket,smallrye-health,elytron-security-properties-file
cd quarkus-aesh-terminal

The core extension finds Aesh commands and runs the console. SSH and WebSocket connect remote sessions to that one command registry. SmallRye Health reports whether those transports are available. The properties-file security extension gives us a small development identity store for the browser endpoint.

Add the Aesh test harness to pom.xml next to the generated test dependencies:

<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-test-aesh</artifactId>
    <scope>test</scope>
</dependency>

Give the terminal application state

The terminal must not create a second path around SwiftShip’s rules. We start with the application service that owns hub state, retry decisions, and traffic changes. Create src/main/java/com/themainthread/terminal/dispatch/DispatchService.java:

package com.themainthread.terminal.dispatch;

import java.util.ArrayList;
import java.util.Comparator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Optional;

import jakarta.enterprise.context.ApplicationScoped;

@ApplicationScoped
public class DispatchService {

    private final Map<String, Hub> hubs = new LinkedHashMap<>();

    public DispatchService() {
        reset();
    }

    public synchronized void reset() {
        hubs.clear();

        Hub berlin = new Hub("berlin", 184, true);
        berlin.failures.put("SHP-1042",
                new ShipmentFailure("SHP-1042", "carrier timeout", 3, "fallback-dhl", ShipmentState.FAILED));
        berlin.failures.put("SHP-1077",
                new ShipmentFailure("SHP-1077", "label rejected", 1, "manual-review", ShipmentState.FAILED));
        hubs.put(berlin.name, berlin);

        hubs.put("madrid", new Hub("madrid", 12, true));
        hubs.put("oslo", new Hub("oslo", 7, true));
    }

    public synchronized List<HubSnapshot> hubs() {
        return hubs.values().stream()
                .map(this::snapshot)
                .sorted(Comparator.comparing(HubSnapshot::name))
                .toList();
    }

    public synchronized Optional<HubSnapshot> hub(String name) {
        return Optional.ofNullable(hubs.get(normalize(name))).map(this::snapshot);
    }

    public synchronized boolean hasHub(String name) {
        return hubs.containsKey(normalize(name));
    }

    public synchronized List<String> hubNames() {
        return hubs.keySet().stream().sorted().toList();
    }

    public synchronized List<ShipmentFailure> failures(String hubName) {
        Hub hub = hubs.get(normalize(hubName));
        if (hub == null) {
            return List.of();
        }
        return hub.failures.values().stream()
                .filter(failure -> failure.state() == ShipmentState.FAILED)
                .sorted(Comparator.comparing(ShipmentFailure::id))
                .toList();
    }

    public synchronized List<String> failedShipmentIds() {
        List<String> ids = new ArrayList<>();
        for (Hub hub : hubs.values()) {
            hub.failures.values().stream()
                    .filter(failure -> failure.state() == ShipmentState.FAILED)
                    .map(ShipmentFailure::id)
                    .forEach(ids::add);
        }
        return ids.stream().sorted().toList();
    }

    public synchronized OperationResult retry(String hubName, String shipmentId, boolean dryRun, String confirmation) {
        Hub hub = hubs.get(normalize(hubName));
        if (hub == null) {
            return OperationResult.failure("Unknown hub '" + hubName + "'.");
        }

        String normalizedId = normalizeId(shipmentId);
        ShipmentFailure failure = hub.failures.get(normalizedId);
        if (failure == null) {
            return OperationResult.failure("Shipment " + normalizedId + " is not part of this hub.");
        }
        if (failure.state() == ShipmentState.RETRY_QUEUED) {
            return OperationResult.success("Shipment " + normalizedId + " is already queued for retry.");
        }

        if (dryRun) {
            return OperationResult.success("Would retry " + normalizedId + " through " + failure.nextRoute() + ".");
        }
        if (!normalizedId.equals(normalizeId(confirmation))) {
            return OperationResult.failure("Retry refused. Pass --confirm=" + normalizedId + ".");
        }

        hub.failures.put(normalizedId, failure.withState(ShipmentState.RETRY_QUEUED));
        hub.queued++;
        return OperationResult.success("Shipment " + normalizedId + " queued through " + failure.nextRoute() + ".");
    }

    public synchronized OperationResult drain(String hubName, String confirmation) {
        Hub hub = hubs.get(normalize(hubName));
        if (hub == null) {
            return OperationResult.failure("Unknown hub '" + hubName + "'.");
        }
        if (!hub.name.equals(normalize(confirmation))) {
            return OperationResult.failure("Drain refused. Pass --confirm=" + hub.name + ".");
        }
        if (!hub.acceptingTraffic) {
            return OperationResult.success("Hub " + hub.name + " is already drained.");
        }

        hub.acceptingTraffic = false;
        return OperationResult.success("Hub " + hub.name + " stopped accepting new traffic.");
    }

    public synchronized OperationResult resume(String hubName, String confirmation) {
        Hub hub = hubs.get(normalize(hubName));
        if (hub == null) {
            return OperationResult.failure("Unknown hub '" + hubName + "'.");
        }
        if (!hub.name.equals(normalize(confirmation))) {
            return OperationResult.failure("Resume refused. Pass --confirm=" + hub.name + ".");
        }
        if (hub.acceptingTraffic) {
            return OperationResult.success("Hub " + hub.name + " is already accepting traffic.");
        }

        hub.acceptingTraffic = true;
        return OperationResult.success("Hub " + hub.name + " resumed traffic.");
    }

    private HubSnapshot snapshot(Hub hub) {
        long failed = hub.failures.values().stream()
                .filter(failure -> failure.state() == ShipmentState.FAILED)
                .count();
        HubState state = !hub.acceptingTraffic ? HubState.DRAINED : failed > 0 ? HubState.DEGRADED : HubState.HEALTHY;
        return new HubSnapshot(hub.name, state, hub.queued, failed, hub.acceptingTraffic);
    }

    private static String normalize(String value) {
        return value == null ? "" : value.trim().toLowerCase(Locale.ROOT);
    }

    private static String normalizeId(String value) {
        return value == null ? "" : value.trim().toUpperCase(Locale.ROOT);
    }

    public enum HubState {
        HEALTHY,
        DEGRADED,
        DRAINED
    }

    public enum ShipmentState {
        FAILED,
        RETRY_QUEUED
    }

    public record HubSnapshot(String name, HubState state, int queued, long failed, boolean acceptingTraffic) {
    }

    public record ShipmentFailure(String id, String reason, int attempts, String nextRoute, ShipmentState state) {
        ShipmentFailure withState(ShipmentState newState) {
            return new ShipmentFailure(id, reason, attempts, nextRoute, newState);
        }
    }

    public record OperationResult(boolean successful, String message) {
        static OperationResult success(String message) {
            return new OperationResult(true, message);
        }

        static OperationResult failure(String message) {
            return new OperationResult(false, message);
        }
    }

    private static final class Hub {
        private final String name;
        private final Map<String, ShipmentFailure> failures = new LinkedHashMap<>();
        private int queued;
        private boolean acceptingTraffic;

        private Hub(String name, int queued, boolean acceptingTraffic) {
            this.name = name;
            this.queued = queued;
            this.acceptingTraffic = acceptingTraffic;
        }
    }
}

The synchronization is enough for this deterministic example. In a real application, the command should call the same transactional service as the normal application path. Direct SQL in the command would bypass the validation and transaction rules we are trying to preserve.

Write the first command

The operator’s first need is orientation, not mutation. status shows every hub and makes Berlin’s degraded state visible before we touch anything. Create src/main/java/com/themainthread/terminal/cli/StatusCommand.java:

package com.themainthread.terminal.cli;

import com.themainthread.terminal.dispatch.DispatchService;

import jakarta.inject.Inject;

import org.aesh.command.Command;
import org.aesh.command.CommandDefinition;
import org.aesh.command.CommandResult;
import org.aesh.command.invocation.CommandInvocation;

@CommandDefinition(name = "status", description = "Show every dispatch hub")
public class StatusCommand implements Command<CommandInvocation> {

    @Inject
    DispatchService dispatchService;

    @Override
    public CommandResult execute(CommandInvocation invocation) {
        invocation.println(String.format("%-10s %-10s %8s %7s %8s", "HUB", "STATE", "QUEUED", "FAILED", "TRAFFIC"));
        dispatchService.hubs().forEach(hub -> invocation.println(String.format(
                "%-10s %-10s %8d %7d %8s",
                hub.name(),
                hub.state(),
                hub.queued(),
                hub.failed(),
                hub.acceptingTraffic() ? "OPEN" : "STOPPED")));
        return CommandResult.SUCCESS;
    }
}

@CommandDefinition gives Aesh the name and help text. Output goes through CommandInvocation to the SSH or browser session that invoked it. Our operator can now inspect Berlin without a database login.

I normally use constructor injection in Quarkus. Aesh command classes are different because Aesh creates their instances first and Quarkus injects fields afterwards. Field injection matches that lifecycle. The application service stays a normal CDI bean.

Add a command group and live input help

status tells us where the problem is. Aesh mixins let the next commands share one --hub target. Create src/main/java/com/themainthread/terminal/cli/HubTarget.java:

package com.themainthread.terminal.cli;

import org.aesh.command.option.Option;

public class HubTarget {

    @Option(
            name = "hub",
            shortName = 'h',
            description = "Hub name",
            required = true,
            completer = HubNameCompleter.class,
            validator = HubNameValidator.class)
    String name;

    String name() {
        return name;
    }
}

The operator should not memorize hub names during an incident. This completer reads them from current application state. Create HubNameCompleter.java:

package com.themainthread.terminal.cli;

import com.themainthread.terminal.dispatch.DispatchService;

import jakarta.enterprise.context.Dependent;
import jakarta.inject.Inject;

import org.aesh.command.completer.CompleterInvocation;
import org.aesh.command.completer.OptionCompleter;

import io.quarkus.arc.Unremovable;

@Dependent
@Unremovable
public class HubNameCompleter implements OptionCompleter<CompleterInvocation> {

    @Inject
    DispatchService dispatchService;

    @Override
    public void complete(CompleterInvocation invocation) {
        String input = invocation.getGivenCompleteValue();
        dispatchService.hubNames().stream()
                .filter(name -> input == null || input.isBlank() || name.startsWith(input))
                .forEach(invocation::addCompleterValue);
    }
}

Create HubNameValidator.java beside it:

package com.themainthread.terminal.cli;

import com.themainthread.terminal.dispatch.DispatchService;

import jakarta.enterprise.context.Dependent;
import jakarta.inject.Inject;

import org.aesh.command.validator.OptionValidator;
import org.aesh.command.validator.OptionValidatorException;
import org.aesh.command.validator.ValidatorInvocation;

import io.quarkus.arc.Unremovable;

@Dependent
@Unremovable
public class HubNameValidator implements OptionValidator<ValidatorInvocation<String, ?>> {

    @Inject
    DispatchService dispatchService;

    @Override
    public void validate(ValidatorInvocation<String, ?> invocation) throws OptionValidatorException {
        if (!dispatchService.hasHub(invocation.getValue())) {
            throw new OptionValidatorException("Unknown hub '" + invocation.getValue() + "'.");
        }
    }
}

The completer suggests live values, while the validator rejects an unknown hub. @Unremovable keeps both helpers available because normal Java call analysis cannot see Aesh’s annotation references.

Apply the same idea to shipment IDs. Completion should offer Berlin’s current failures, not require a copy from another screen. Create FailedShipmentCompleter.java:

package com.themainthread.terminal.cli;

import com.themainthread.terminal.dispatch.DispatchService;

import jakarta.enterprise.context.Dependent;
import jakarta.inject.Inject;

import org.aesh.command.completer.CompleterInvocation;
import org.aesh.command.completer.OptionCompleter;

import io.quarkus.arc.Unremovable;

@Dependent
@Unremovable
public class FailedShipmentCompleter implements OptionCompleter<CompleterInvocation> {

    @Inject
    DispatchService dispatchService;

    @Override
    public void complete(CompleterInvocation invocation) {
        String input = invocation.getGivenCompleteValue();
        dispatchService.failedShipmentIds().stream()
                .filter(id -> input == null || input.isBlank() || id.startsWith(input.toUpperCase()))
                .forEach(invocation::addCompleterValue);
    }
}

Create HubCommand.java. It registers four operations, reuses HubTarget, and adds safety and audit behavior to state changes:

package com.themainthread.terminal.cli;

import com.themainthread.terminal.audit.CommandAudit;
import com.themainthread.terminal.dispatch.DispatchService;
import com.themainthread.terminal.dispatch.DispatchService.OperationResult;

import jakarta.inject.Inject;

import org.aesh.command.Command;
import org.aesh.command.CommandDefinition;
import org.aesh.command.CommandResult;
import org.aesh.command.invocation.CommandInvocation;
import org.aesh.command.option.Argument;
import org.aesh.command.option.Mixin;
import org.aesh.command.option.Option;

@CommandDefinition(
        name = "hub",
        description = "Operate one dispatch hub",
        groupCommands = {
                FailuresCommand.class,
                RetryCommand.class,
                DrainCommand.class,
                ResumeCommand.class
        })
public class HubCommand implements Command<CommandInvocation> {

    @Override
    public CommandResult execute(CommandInvocation invocation) {
        invocation.println("Choose a subcommand: failures, retry, drain, or resume.");
        return CommandResult.SUCCESS;
    }
}

@CommandDefinition(name = "failures", description = "List failed shipments")
class FailuresCommand implements Command<CommandInvocation> {

    @Mixin
    HubTarget target;

    @Inject
    DispatchService dispatchService;

    @Override
    public CommandResult execute(CommandInvocation invocation) {
        var failures = dispatchService.failures(target.name());
        if (failures.isEmpty()) {
            invocation.println("No failed shipments.");
            return CommandResult.SUCCESS;
        }

        invocation.println(String.format("%-10s %-18s %8s %-16s", "SHIPMENT", "REASON", "ATTEMPTS", "NEXT ROUTE"));
        failures.forEach(failure -> invocation.println(String.format(
                "%-10s %-18s %8d %-16s",
                failure.id(), failure.reason(), failure.attempts(), failure.nextRoute())));
        return CommandResult.SUCCESS;
    }
}

@CommandDefinition(name = "retry", description = "Preview or queue one retry")
class RetryCommand implements Command<CommandInvocation> {

    @Mixin
    HubTarget target;

    @Argument(description = "Shipment ID", required = true, completer = FailedShipmentCompleter.class)
    String shipmentId;

    @Option(name = "dry-run", description = "Show the retry plan without changing state", hasValue = false)
    boolean dryRun;

    @Option(name = "confirm", description = "Repeat the shipment ID to authorize the retry")
    String confirmation;

    @Inject
    DispatchService dispatchService;

    @Inject
    CommandAudit commandAudit;

    @Override
    public CommandResult execute(CommandInvocation invocation) {
        long started = System.nanoTime();
        String hubName = target.name();
        OperationResult result = dispatchService.retry(hubName, shipmentId, dryRun, confirmation);
        invocation.println(result.message());
        CommandResult commandResult = result.successful() ? CommandResult.SUCCESS : CommandResult.FAILURE;
        commandAudit.record(
                "retry " + shipmentId + " in " + hubName + (dryRun ? " (dry-run)" : ""),
                commandResult,
                (System.nanoTime() - started) / 1_000_000);
        return commandResult;
    }
}

@CommandDefinition(name = "drain", description = "Stop new traffic for the selected hub")
class DrainCommand implements Command<CommandInvocation> {

    @Mixin
    HubTarget target;

    @Option(name = "confirm", description = "Repeat the hub name to authorize the drain", required = true)
    String confirmation;

    @Inject
    DispatchService dispatchService;

    @Inject
    CommandAudit commandAudit;

    @Override
    public CommandResult execute(CommandInvocation invocation) {
        long started = System.nanoTime();
        String hubName = target.name();
        OperationResult result = dispatchService.drain(hubName, confirmation);
        invocation.println(result.message());
        CommandResult commandResult = result.successful() ? CommandResult.SUCCESS : CommandResult.FAILURE;
        commandAudit.record(
                "drain " + hubName,
                commandResult,
                (System.nanoTime() - started) / 1_000_000);
        return commandResult;
    }
}

@CommandDefinition(name = "resume", description = "Resume traffic for the selected hub")
class ResumeCommand implements Command<CommandInvocation> {

    @Mixin
    HubTarget target;

    @Option(name = "confirm", description = "Repeat the hub name to authorize the resume", required = true)
    String confirmation;

    @Inject
    DispatchService dispatchService;

    @Inject
    CommandAudit commandAudit;

    @Override
    public CommandResult execute(CommandInvocation invocation) {
        long started = System.nanoTime();
        String hubName = target.name();
        OperationResult result = dispatchService.resume(hubName, confirmation);
        invocation.println(result.message());
        CommandResult commandResult = result.successful() ? CommandResult.SUCCESS : CommandResult.FAILURE;
        commandAudit.record(
                "resume " + hubName,
                commandResult,
                (System.nanoTime() - started) / 1_000_000);
        return commandResult;
    }
}

hub failures turns “Berlin is degraded” into exact shipments. retry --dry-run shows the route before changing state. Confirmed retries repeat the shipment ID; drain and resume repeat the hub name. Incomplete commands fail before they change live traffic.

Typing hub enters a group context. I use explicit commands such as hub retry --hub=berlin … because they are easier to test and copy into an incident record.

Audit the state changes

After Berlin recovers, somebody will ask what changed. We record retry, drain, and resume results at the command boundary, keeping the last 100 in memory.

Create src/main/java/com/themainthread/terminal/audit/CommandAudit.java:

package com.themainthread.terminal.audit;

import java.time.Instant;
import java.util.List;
import java.util.concurrent.ConcurrentLinkedDeque;
import java.util.regex.Pattern;

import jakarta.enterprise.context.ApplicationScoped;

import org.aesh.command.CommandResult;
import org.jboss.logging.Logger;

@ApplicationScoped
public class CommandAudit {

    private static final Logger LOG = Logger.getLogger(CommandAudit.class);
    private static final int MAX_ENTRIES = 100;
    private static final Pattern SENSITIVE_OPTION = Pattern.compile(
            "(?i)(--(?:password|secret|token)(?:=|\\s+))\\S+");

    private final ConcurrentLinkedDeque<AuditEntry> entries = new ConcurrentLinkedDeque<>();

    public void record(String commandLine, CommandResult result, long executionTimeMs) {
        String safeCommandLine = SENSITIVE_OPTION.matcher(commandLine).replaceAll("$1***");
        entries.addFirst(new AuditEntry(Instant.now(), safeCommandLine, result, executionTimeMs));
        while (entries.size() > MAX_ENTRIES) {
            entries.pollLast();
        }
        LOG.infof("Aesh command completed: command='%s', result=%s, durationMs=%d",
                safeCommandLine, result, executionTimeMs);
    }

    public List<AuditEntry> recent(int limit) {
        return entries.stream().limit(limit).toList();
    }

    public record AuditEntry(Instant timestamp, String commandLine, CommandResult result, long executionTimeMs) {
    }
}

Our operator also needs to inspect that record from the same command vocabulary. Create src/main/java/com/themainthread/terminal/cli/AuditCommand.java:

package com.themainthread.terminal.cli;

import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;

import com.themainthread.terminal.audit.CommandAudit;

import jakarta.inject.Inject;

import org.aesh.command.Command;
import org.aesh.command.CommandDefinition;
import org.aesh.command.CommandResult;
import org.aesh.command.invocation.CommandInvocation;
import org.aesh.command.option.Option;

@CommandDefinition(name = "audit", description = "Show recent command executions")
public class AuditCommand implements Command<CommandInvocation> {

    private static final DateTimeFormatter TIMESTAMP = DateTimeFormatter
            .ofPattern("uuuu-MM-dd HH:mm:ss'Z'")
            .withZone(ZoneOffset.UTC);

    @Option(name = "limit", shortName = 'n', description = "Maximum entries", defaultValue = "10")
    int limit;

    @Inject
    CommandAudit commandAudit;

    @Override
    public CommandResult execute(CommandInvocation invocation) {
        var entries = commandAudit.recent(Math.max(1, Math.min(limit, 50)));
        if (entries.isEmpty()) {
            invocation.println("No commands recorded in this process.");
            return CommandResult.SUCCESS;
        }

        invocation.println(String.format("%-20s %-8s %8s %s", "TIME", "RESULT", "MS", "COMMAND"));
        entries.forEach(entry -> invocation.println(String.format(
                "%-20s %-8s %8d %s",
                TIMESTAMP.format(entry.timestamp()),
                entry.result().isSuccess() ? "SUCCESS" : "FAILURE",
                entry.executionTimeMs(),
                entry.commandLine())));
        return CommandResult.SUCCESS;
    }
}

The deque proves the interaction without a database. A real service should emit a durable event with the authenticated subject, target, action, result, and correlation ID. Process memory disappears during restarts, often just before the incident review.

We also need to know when remote sessions open and close. Aesh emits asynchronous events for both transports. Create src/main/java/com/themainthread/terminal/audit/SessionAudit.java:

package com.themainthread.terminal.audit;

import io.quarkus.aesh.runtime.AeshSessionEvent;
import io.quarkus.aesh.runtime.SessionClosed;
import io.quarkus.aesh.runtime.SessionOpened;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.event.ObservesAsync;

import org.jboss.logging.Logger;

@ApplicationScoped
public class SessionAudit {

    private static final Logger LOG = Logger.getLogger(SessionAudit.class);

    void opened(@ObservesAsync @SessionOpened AeshSessionEvent event) {
        LOG.infof("Aesh session opened: id=%s, transport=%s, timestamp=%s",
                event.sessionId(), event.transport(), event.timestamp());
    }

    void closed(@ObservesAsync @SessionClosed AeshSessionEvent event) {
        LOG.infof("Aesh session closed: id=%s, transport=%s, timestamp=%s",
                event.sessionId(), event.transport(), event.timestamp());
    }
}

Each event has a session ID, transport, and timestamp. It does not solve per-user authorization because the SSH username does not become a Quarkus SecurityIdentity.

Secure and limit both transports

Both transports open a path into SwiftShip. Before the operator connects, add authentication, session limits, idle timeouts, and deliberate network bindings in src/main/resources/application.properties:

quarkus.aesh.mode=console
quarkus.aesh.prompt=[swiftship]$ 
quarkus.aesh.start-console=false
quarkus.aesh.add-exit-command=true
quarkus.aesh.persist-history=false

quarkus.aesh.websocket.roles-allowed=operator
quarkus.aesh.websocket.max-connections=4
quarkus.aesh.websocket.idle-timeout=15m

quarkus.aesh.ssh.host=127.0.0.1
quarkus.aesh.ssh.port=2222
quarkus.aesh.ssh.authorized-keys-file=${SSH_AUTHORIZED_KEYS_FILE:target/aesh/authorized_keys}
quarkus.aesh.ssh.host-key-file=${SSH_HOST_KEY_FILE:target/aesh/host-key.ser}
quarkus.aesh.ssh.max-connections=4
quarkus.aesh.ssh.idle-timeout=15m

quarkus.http.auth.basic=true
quarkus.http.auth.realm=SwiftShip Operations
quarkus.http.auth.policy.operator.roles-allowed=operator
quarkus.http.auth.permission.aesh.paths=/aesh/*
quarkus.http.auth.permission.aesh.policy=operator

%dev.quarkus.security.users.embedded.enabled=true
%dev.quarkus.security.users.embedded.plain-text=true
%dev.quarkus.security.users.embedded.users.operator=terminal
%dev.quarkus.security.users.embedded.roles.operator=operator
%dev.quarkus.security.users.embedded.users.viewer=terminal
%dev.quarkus.security.users.embedded.roles.viewer=viewer

%test.quarkus.security.users.embedded.enabled=true
%test.quarkus.security.users.embedded.plain-text=true
%test.quarkus.security.users.embedded.users.operator=terminal
%test.quarkus.security.users.embedded.roles.operator=operator
%test.quarkus.security.users.embedded.users.viewer=terminal
%test.quarkus.security.users.embedded.roles.viewer=viewer
%test.quarkus.aesh.ssh.enabled=false
%test.quarkus.aesh.start-console=true

Remote transports force console mode. The start-console=false setting leaves the local terminal to the Quarkus dev console. We disable persistent command history because it may contain shipment identifiers or secrets.

SSH and WebSocket share commands, not authentication. The WebSocket endpoint requires the operator role, and the HTTP permission protects everything below /aesh/*. The viewer account proves that a signed-in user without the role receives 403.

The embedded realm only exists in the dev and test profiles. The Quarkus properties-file security guide limits this mechanism to development and testing. Production has no embedded users, so access stays closed until you connect a real identity provider.

SSH checks its own authorized keys. By default, Aesh has no connection cap or idle timeout, so we set both and bind SSH to loopback. A terminal forgotten after the Berlin incident should not stay connected forever.

Generate a key the current stack accepts

For this tutorial, SSH should reject anyone who does not hold the operator key. Generate it after a clean build because target is disposable:

mkdir -p target/aesh
ssh-keygen -q -t rsa -b 3072 -N '' -f target/aesh/operator
cp target/aesh/operator.pub target/aesh/authorized_keys

RSA is intentional. I first used the modern Ed25519 default. The Quarkus 3.39.1 Aesh SSH stack rejected it with EdDSA provider not supported and then asked for a password. A 3072-bit RSA key succeeds with the current stack. Check this again after the SSH support changes. I haven’t checked if this has been filed as an issue or not.

Start Quarkus:

./mvnw quarkus:dev

The application starts as a normal server. The Quarkus dev console keeps the local terminal, while Aesh accepts SSH and WebSocket sessions. We can now deal with Berlin through the vocabulary we defined.

Operate Berlin over SSH

Berlin is still degraded, and the operator key is ready. Open another terminal and connect:

ssh -tt -i target/aesh/operator \
  -p 2222 \
  -o IdentitiesOnly=yes \
  -o StrictHostKeyChecking=accept-new \
  -o UserKnownHostsFile=target/aesh/known_hosts \
  operator@127.0.0.1

Start with status. It gives us orientation before we change anything:

[swiftship]$ status
HUB        STATE        QUEUED  FAILED  TRAFFIC
berlin     DEGRADED        184       2     OPEN
madrid     HEALTHY          12       0     OPEN
oslo       HEALTHY           7       0     OPEN

Berlin alone is degraded and still accepts traffic. Narrow the problem to its failed shipments:

[swiftship]$ hub failures --hub=berlin
SHIPMENT   REASON             ATTEMPTS NEXT ROUTE
SHP-1042   carrier timeout           3 fallback-dhl
SHP-1077   label rejected            1 manual-review

We can see both failures and their routes. Press Tab after --hub= to complete a live hub, then press Tab in the shipment ID to complete a current failure. DispatchService supplies the values at runtime, so the operator does not memorize them.

SHP-1042 has already failed three times, so we inspect the next action before asking SwiftShip to take it:

[swiftship]$ hub retry --hub=berlin SHP-1042 --dry-run
Would retry SHP-1042 through fallback-dhl.

The dry run shows fallback-dhl and leaves state unchanged. That is the route we expected, so confirm the retry by repeating the shipment ID:

[swiftship]$ hub retry --hub=berlin SHP-1042 --confirm=SHP-1042
Shipment SHP-1042 queued through fallback-dhl.
[swiftship]$ hub retry --hub=berlin SHP-1042 --confirm=SHP-1042
Shipment SHP-1042 is already queued for retry.

The first call queues the retry. The second returns success without another queue entry. Operators repeat commands when a terminal pauses, so idempotency belongs in the service. Inspect the record:

[swiftship]$ audit --limit=3
TIME                 RESULT         MS COMMAND
2026-08-28 07:23:17Z SUCCESS         0 retry SHP-1042 in berlin
2026-08-28 07:23:17Z SUCCESS         1 retry SHP-1042 in berlin (dry-run)

The audit shows the preview and accepted retry. Berlin still accepts new traffic. If the carrier remains unstable, we may need to stop more shipments from entering the queue. Without confirmation, Aesh refuses the command before execution; with it, the service verifies Berlin:

[swiftship]$ hub drain --hub=berlin --confirm=berlin
Hub berlin stopped accepting new traffic.
[swiftship]$ status
HUB        STATE        QUEUED  FAILED  TRAFFIC
berlin     DRAINED         185       1  STOPPED
madrid     HEALTHY          12       0     OPEN
oslo       HEALTHY           7       0     OPEN

SHP-1042 is now queued through the known fallback route, one failure remains, and Berlin accepts no new traffic. The carrier is not repaired, but SwiftShip is in a deliberate state. The operator never touched Linux or the database.

Use hub resume --hub=berlin --confirm=berlin when the carrier is ready for new traffic, then exit to close the SSH session.

Use the same console in a browser

SSH is one path to the commands, not a separate interface. Open http://localhost:8080/aesh/index.html and sign in with the development-only user operator and password terminal. The local xterm.js client needs no CDN.

The browser uses the same Aesh registry, prompt, completion, history, and commands as SSH. We maintain one admin interface and choose the transport that fits the access path.

Check transport readiness

The operator depends on these transports during an incident, so we need a machine-readable startup signal. The extensions publish readiness when quarkus-smallrye-health is present:

curl -s http://127.0.0.1:8080/q/health/ready

The response includes both transports and their current connection counts:

{
  "status": "UP",
  "checks": [
    {
      "name": "Aesh WebSocket terminal health check",
      "status": "UP",
      "data": {
        "path": "/aesh/terminal",
        "activeConnections": 0
      }
    },
    {
      "name": "Aesh SSH server health check",
      "status": "UP",
      "data": {
        "host": "127.0.0.1",
        "port": 2222,
        "activeConnections": 0
      }
    }
  ]
}

This proves that both transports started and reports their connections. It does not prove that fallback-dhl can accept SHP-1042 or that Berlin is healthy. Transport readiness and business readiness answer different questions.

Test commands, state, and access

The SSH sequence proves the path by hand. Automated tests protect it before an upgrade or command change reaches an operator. The Aesh harness drives the real read-evaluate-print loop (REPL) and checks its output. Create src/test/java/com/themainthread/terminal/TerminalCommandTest.java:

package com.themainthread.terminal;

import static org.junit.jupiter.api.Assertions.assertTrue;

import org.junit.jupiter.api.Test;

import io.quarkus.test.aesh.AeshLauncher;
import io.quarkus.test.junit.main.QuarkusMainTest;

@QuarkusMainTest
class TerminalCommandTest {

    @Test
    void operatesAHubThroughTheRepl(AeshLauncher launcher) {
        launcher.launch();
        try {
            String status = launcher.executeCommand("status");
            assertTrue(status.contains("berlin"));
            assertTrue(status.contains("DEGRADED"));

            String dryRun = launcher.executeCommand("hub retry --hub=berlin SHP-1042 --dry-run");
            assertTrue(dryRun.contains("Would retry SHP-1042 through fallback-dhl"), dryRun);

            String retry = launcher.executeCommand("hub retry --hub=berlin SHP-1042 --confirm=SHP-1042");
            assertTrue(retry.contains("Shipment SHP-1042 queued through fallback-dhl"), retry);

            String repeated = launcher.executeCommand("hub retry --hub=berlin SHP-1042 --confirm=SHP-1042");
            assertTrue(repeated.contains("already queued for retry"), repeated);

            String audit = launcher.executeCommand("audit --limit=3");
            assertTrue(audit.contains("retry SHP-1042 in berlin"), audit);
        } finally {
            launcher.exit();
        }
    }
}

Create src/test/java/com/themainthread/terminal/TerminalAccessTest.java for the browser policy and readiness contract. It checks the three cases we care about: anonymous, authenticated without the role, and operator:

package com.themainthread.terminal;

import static io.restassured.RestAssured.given;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasItem;

import org.junit.jupiter.api.Test;

import io.quarkus.test.junit.QuarkusTest;

@QuarkusTest
class TerminalAccessTest {

    @Test
    void protectsTheBrowserTerminal() {
        given()
                .redirects().follow(false)
                .when().get("/aesh/index.html")
                .then().statusCode(401);

        given()
                .auth().preemptive().basic("viewer", "terminal")
                .when().get("/aesh/index.html")
                .then().statusCode(403);

        given()
                .auth().preemptive().basic("operator", "terminal")
                .when().get("/aesh/index.html")
                .then().statusCode(200);
    }

    @Test
    void publishesTransportReadiness() {
        given()
                .when().get("/q/health/ready")
                .then()
                .statusCode(200)
                .body("status", equalTo("UP"))
                .body("checks.name", hasItem("Aesh WebSocket terminal health check"));
    }
}

A service test also covers the safeguards we used during the Berlin sequence: confirmation and idempotency. Run everything:

./mvnw test

The verified result is:

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS

Decide whether this belongs in production

SwiftShip still uses deterministic state, development users, and disposable keys. Before trusting this terminal during a real incident, review the boundary around its commands.

All three Aesh extensions are preview features. Pin the Quarkus version and test every upgrade. The rejected Ed25519 key already shows one problem.

HTTP and SSH authentication remain separate. WebSocket uses Quarkus Security and can enforce roles. SSH checks passwords or authorized keys, but it cannot map that login to a SecurityIdentity. Every accepted key reaches the same commands here. Add another access layer for authorization per person or command.

Replace the development credentials and paths. Use a real HTTP identity provider. Mount authorized_keys and the host key from managed storage, with defined rotation and revocation.

Keep SSH behind a controlled access layer. Restrict /aesh/*, /aesh/terminal, and port 2222. Configure browser origins because WebSocket lacks normal same-origin protection. Move the in-memory audit to durable storage outside the JVM.

Keep commands narrow and make reads the default. Mutations need authorization and validation. Add dry runs, confirmation, idempotency, timeouts, and application audit events where the operation needs them. A wrapped shell command gives none of these guarantees.

The terminal is also attached to one application instance. Its view and drain may be local only. Show that scope in each command or call a shared control service with a clear cluster contract.

Conclusion

I started this with a slightly uncomfortable idea: putting a terminal back into a running application after spending years trying to get shells out of production.

After building it, I think the distinction is pretty clear. The interesting part is not the terminal but the bounded operations interface behind it. SwiftShip exposes five commands that understand application state, call the normal business services, validate their input, require confirmation for dangerous changes, and leave evidence behind. SSH and the browser are only transports.

That boundary is also what decides whether this belongs anywhere near production. A general shell is still a bad idea. So is hiding Runtime.exec(), SQL, or arbitrary HTTP calls behind friendly command names. Keep the vocabulary small, keep mutations explicit, use real identities, persist the audit trail outside the JVM, and be very clear about whether a command affects one application instance or the whole system. The Aesh extensions are also still preview features, so this is something I would introduce deliberately, with the same review you would give any other administrative surface.

For the Berlin incident, that meant the operator could inspect two failures, preview a retry, queue it safely, and drain incoming traffic. No container shell. No database login. No second admin API to maintain.

Five commands were enough. And that is exactly the point.

Discussion about this post

Ready for more?