# Testing

The server has ~850 `@Test` methods across ~80 files. The naming is mostly self-explanatory and the test support layer is small — but a few decisions are worth knowing about up front, especially the deliberate absence of any forked-JVM tests in CI.

## 1. The split

There are four logical tiers of tests, all running under JUnit 5:

| Tier | Examples | Approach |
|---|---|---|
| **Pure unit** | `BreakpointTrackerTest.PendingBreakpoints` (`BreakpointTrackerTest.java:32-143`), `EvaluationGuardTest`, `EventHistoryTest`, `ThreadFormattingTest`, `JDWPToolsParseCharInputTest`, `JdwpAgentArgParserTest`, `DiagnoseReportRendererTest` | No JDI types involved. Plain JUnit assertions against pojo behaviour. |
| **Mocked-JDI unit** (the bulk) | every `JdiEventListener*Test`, `JDIConnectionService*Test`, `JDWPTools*Test` | `BreakpointRequest`, `ThreadReference`, `EventSet`, `Location`, etc. mocked via Mockito. |
| **In-process integration** | `InMemoryJavaCompilerPackageIntegrationTest` (`evaluation/InMemoryJavaCompilerPackageIntegrationTest.java:36`), `JdwpHandshakeProbeTest`, `JvmDiscoveryServiceConfirmAllTest` | Runs real Eclipse JDT against a tiny in-process classpath, or talks to a `FakeJdwpServer` over a loopback socket. |
| **Sandbox** | The `jdwp-sandbox/` module | Deliberately broken classes designed to fail under JDWP. Skipped by default. |

**No test forks a real Java process or attaches to a real JDWP port.** `ripgrep`-ing the test sources shows zero uses of `ProcessBuilder` or `AttachingConnector.attach`. The closest thing to "real network" is `FakeJdwpServer` — an in-process loopback `ServerSocket` that returns the 14-byte JDWP handshake.

Live-JVM verification is done **manually** via the sandbox module, not automatically in CI. See § 4.

## 2. Mocking JDI

> **Background — why testing JDI is hard**
>
> JDI is one of the rare Java APIs with no first-party in-process fake. `com.sun.jdi.VirtualMachine` and its dozens of friends (`ThreadReference`, `ObjectReference`, `ReferenceType`, `StackFrame`, `Location`, `Method`, `Field`, every event subtype) are all interfaces, but every concrete implementation in OpenJDK lives in `com.sun.tools.jdi` and is tied to a real JDWP socket — there is no `LocalVirtualMachine` or "in-memory backend" that a test could spin up. Constructing them requires either a forked target JVM or substantial reflection.
>
> The two practical options are: (a) fork a real Java process and attach to it via `SocketAttachingConnector`, or (b) mock the interfaces. Option (a) is heavy (each test pays a 1–3 s JVM startup), brittle (timing, port conflicts, OS differences), and complicates CI. Option (b) is fast and deterministic but tedious — JDI chains are long (`event.thread().referenceType().name()`) and each link needs a stub.
>
> This codebase commits hard to option (b). The support classes below exist purely to keep the mock-chain setup compact.

Every JDI type is mocked with Mockito. Three test-support classes centralise the patterns so individual tests stay short:

### `JdiEventListenerTestSupport`

`JdiEventListenerTestSupport.java:47`. Factory methods for the event objects the listener consumes:

- `mockEventSet` (line 147)
- `mockThread` (line 153)
- `mockBreakpointEvent` (line 166)
- `mockStepEvent` (line 184)
- `mockExceptionEvent` (lines 203, 213)
- `mockException` (line 247)

Each factory wires the JDI `Location → ReferenceType → name()` chain that the production code walks. A test method composes them into a believable `EventSet` rather than rebuilding the chain from scratch.

### `JDIConnectionServiceTestSupport`

`JDIConnectionServiceTestSupport.java:36`. Provides:

- `newServiceWithMocks` (line 47), `newServiceWithCollaborators` (lines 63, 74) — factory methods.
- `setVm(service, mockVm)` (line 103) — **plants a mocked `VirtualMachine` into the private `vm` field by reflection**, bypassing the real attach path. This is what lets `JDIConnectionService*Test` exercise tool-level behaviour without ever calling `connector.attach(args)`.
- `setLastSuccessfulAttach` (line 118) — plants `lastHost` / `lastPort` for auto-reconnect tests.

### `JDWPToolsTestSupport`

`JDWPToolsTestSupport.java:28`. Factory for the seven-argument `JDWPTools` constructor — it has a lot of collaborators, and the support class assembles them with mocks for unit tests.

### `TestReflectionUtils`

`TestReflectionUtils.java:24` is a single-method helper used to invoke private methods (e.g. `getCollectionView`) on services. It is not the dominant pattern — most reflection happens inline in the support classes. New tests should reach for the support classes first.

## 3. Testing the event listener without a real event queue

`JdiEventListenerTestSupport.runListenerWith` (line 60) is the pattern for driving the listener through a synthesised event:

1. Stub `EventQueue.remove()` against a `BlockingQueue<Object>` pre-loaded with one caller-supplied `EventSet` followed by a `VMDisconnectedException` sentinel.
2. Start the listener.
3. Wait on a `CountDownLatch(2)` for both `EventQueue.remove()` takes.
4. Sleep 30 ms (line 84) to let the listener's catch block settle before assertions.

The sentinel exits the listener cleanly. The companion `runListenerWithEventThenDeathEvent` (line 100) substitutes a synthesised `VMDeathEvent` / `VMDisconnectEvent` for the sentinel, exercising the in-loop graceful-death branch instead of the disconnect-exception branch.

Used end-to-end in `JdiEventListenerEvaluationSuppressionTest.java:75` and many others. Tests assert via `tracker.getLastBreakpointThread()`, `verify(eventSet).resume()`, and `assertLatestEventType` (`JdiEventListenerTestSupport.java:262`) which **deliberately ignores the trailing harness-induced `VM_DEATH` entry**.

The 30 ms sleep is the only timing dependency in the test suite. It is a known flake hotspot if CI is heavily loaded; if you see intermittent failures in `JdiEventListener*Test`, that is the place to look first.

## 4. The sandbox module

`jdwp-sandbox/` is a separate Maven module of **deliberately broken** Java classes. Each one has a paired test that is **expected to fail** when run normally — the failure is the exercise. Examples:

- `bank/TransferService.java:21` — non-atomic transfer with a mid-state audit snapshot. Test "money invariant" fails because the audit captures the intermediate state.
- `order/OrderProcessor.java:29` — silently ignores unknown items. Test catches the dropped data.
- `recursion/RecursiveCalculator.java` — actually correct! Used to verify the recursive-breakpoint protection works end-to-end with a real JVM.

The full set is documented in the top-level [`README.md`](../README.md) as the "test flights".

### Skip-by-default

`jdwp-sandbox/pom.xml:25-34` declares `<skipTests>true</skipTests>` as a **project property** (not surefire config). The reactor-level default `./mvnw test` skips them, but a CLI override works:

```bash
./mvnw -pl jdwp-sandbox test -DskipTests=false
```

This keeps a plain `./mvnw test` green while leaving the sandbox runnable on demand — typically under JDWP, for manual verification of the MCP server against a real target.

The property-not-surefire-config distinction matters: it survives surefire upgrades and works with any plugin that respects `skipTests`.

## 5. Test-naming conventions

- **`XxxTest`** — standard unit / mock test. The default.
- **`XxxLiveTest`** — exercises a real production code path with a hand-built mock object graph (e.g. `JDIConnectionServiceCollectionViewLiveTest.java:47-233` builds a fake HashMap from `mock(ObjectReference.class)`). Despite the name, **there is no real JVM**. "Live" means "exercising the live production code path against realistic mock state". A misleading name, but the convention is what it is.
- **`XxxChainTest`** — tests for the breakpoint-chaining feature (`BreakpointTrackerChainTest`, `JdiEventListenerChainTest`).
- **`XxxIntegrationTest`** — single example, `InMemoryJavaCompilerPackageIntegrationTest`. Actually invokes ECJ for end-to-end compilation. New tests that genuinely cross a real boundary should follow this name.
- **`XxxTestSupport`** — shared scaffolding / factories. Package-private, `final` with private constructor.
- **`Fake*`** — in-process fakes (currently only `FakeJdwpServer`).

Many tests use nested `@Nested` classes named after behavioural areas (`PendingBreakpoints`, `ActiveBreakpoints`, `SuspendPolicy` in `BreakpointTrackerTest.java:34, 147, 176, 274, 349`). The pattern lets one file own a coherent slice of behaviour without becoming a 3,000-line linear scroll.

## 6. Compile-time enforcement

NullAway runs as part of `javac` via Error Prone. The configuration is in `jdwp-mcp-server/pom.xml`:

- Line 21-26 — declares `org.jspecify:jspecify:1.0.0`.
- Line 92 — the Error Prone plugin arg: `-Xplugin:ErrorProne -Xep:NullAway:ERROR -XepOpt:NullAway:OnlyNullMarked=true -XepOpt:NullAway:JSpecifyMode=true -XepExcludedPaths:.*/src/test/.*`. NullAway runs as ERROR (not warning), respects `@NullMarked`, and **excludes tests** from the check.
- Lines 107-118 — `annotationProcessorPaths` registers `error_prone_core` and `nullaway`.
- Line 84 — `<fork>true</fork>` so `-J--add-exports` / `--add-opens` (lines 96-105) reach the forked `javac`.

`@NullMarked` applies at every production package via the per-package `package-info.java` files (root + `evaluation`, `evaluation/exceptions`, `watchers`, `discovery`, `marks`, `transport`). Tests are intentionally not annotated — `@NullMarked` would noisily warn on every Mockito stub that returns `null`.

This means a `./mvnw -pl jdwp-mcp-server compile` that emits any `[NullAway]` diagnostic fails the build. Treat NullAway output as compile errors.

The `jdwp-sandbox` module is intentionally **not** `@NullMarked` — its classes are deliberately broken debugging targets and nullability annotations would muddy the exercises.

## 7. Coverage gaps worth knowing

Some areas are deliberately under-covered:

- **No automated live-JVM coverage in CI.** The full `JDIConnectionService.connect()` path, the JDI event-queue drainer against real JDI, and `RemoteCodeExecutor.defineClass()` injection are never exercised against a real target VM in CI. Verification is the sandbox — manual.
- **`transport/` is untested.** `MultiVersionStdioServerTransportProvider.java` and `StdioTransportConfig.java` have no test files. The override is small and the upstream behaviour is what we're working around, so the absence is defensible — but a smoke test that asserts `protocolVersions()` returns four entries would be cheap and useful.
- **`marks/` and `watchers/` have no dedicated test directories.** Coverage is indirect via `JDWPToolsMarkInstanceTest`, `JDIConnectionServiceMarkLifecycleTest`, `JDWPToolsAttachWatcherTest`, etc.
- **`ClasspathDiscoverer` and `RemoteCodeExecutor` have no `*Test` files** under `evaluation/`. Only `InMemoryJavaCompiler*`, `JdiExpressionEvaluator{GetDeclaredType, Rewrite}Test`, and `JdkDiscoveryService*Test` are present. Both `ClasspathDiscoverer` and `RemoteCodeExecutor` cross the JDWP wire, so unit-testing them well requires a substantial mock harness — the team has so far preferred to exercise them via the sandbox.
- **`JDWPMcpServerApplication`** has no smoke test. It is a one-line `SpringApplication.run`, so the value would be low, but a "context loads" test would catch broken Spring wiring early.

If you're adding code in any of those areas, consider adding the test that wasn't there. The patterns in the support classes will make it cheap.

## 8. Running the tests

The day-to-day commands:

```bash
# Everything green by default
./mvnw test

# Just the MCP server (skips the sandbox module entirely)
./mvnw -pl jdwp-mcp-server test

# Run the sandbox test flights manually (most will fail by design)
./mvnw -pl jdwp-sandbox test -DskipTests=false

# A specific sandbox flight under JDWP, suspended on port 5005
./mvnw -pl jdwp-sandbox test -Dtest=RecursiveCalculatorTest \
       -DskipTests=false -Dmaven.surefire.debug
```

The `-Dmaven.surefire.debug` form starts the test JVM with `-agentlib:jdwp=...,address=5005,suspend=y` — the standard recipe for attaching the MCP server to a sandbox target.

## 9. Adding a test

The shape of a typical test in this codebase:

```java
class FeatureTest {

    private final TestSupport support = new TestSupport();

    @Test
    void description_of_behaviour() {
        // 1. Arrange — build mocks via the support class.
        EventSet eventSet = support.mockEventSet(...);

        // 2. Act — drive the production code.
        listener.handle(eventSet);

        // 3. Assert — read state through the tracker / history / mock verify.
        assertThat(tracker.getLastBreakpointThread()).isEqualTo(...);
    }
}
```

If your test needs JDI types, reach for the support class first. If your test needs a real classpath (e.g. an ECJ-compilation test), look at `InMemoryJavaCompilerPackageIntegrationTest` for the pattern. If your test needs a real JDWP target, you are writing a sandbox test flight, not a unit test — put it in `jdwp-sandbox` and run it manually.

The rule is "tests fast, fast tests" — anything that needs a real JVM goes to the sandbox. The unit suite must stay snappy.
