test: add Forge server characterization

Run JUnit after FML server startup and gate the task on fresh XML because Forge can exit zero after a test-mod exception.
This commit is contained in:
Algent committed 2026-08-14 17:27:18 +02:00
1 parent e33d3b2493
commit 4ae2a6d0fa
5 files changed
+278 -3

No files matched your search

+16 -2
View File
@@ -42,6 +42,16 @@ Allowed divergences are:
Each divergence should record the old behavior, new behavior, reason, compatibility impact, and how it was manually verified. Unintentional behavior changes are regressions even if the new implementation appears cleaner.
### Java runtime policy
The main GTNH release targets modern Java across the Java 17–26 range. A Java 8 artifact must remain available for legacy launchers, with the same functional behavior even if it cannot match the modern runtime's performance.
- Modern Java syntax and APIs may be used when the GTNH build's JVM Downgrader/desugaring path provides a verified Java 8 artifact.
- Do not enable desugaring speculatively. Introduce it with the first source change that needs it, then verify both the modern and downgraded artifacts.
- Java 8 startup, serialization, networking, registration, and gameplay semantics are compatibility requirements. Equal throughput or allocation behavior between Java 8 and modern Java is not required.
- Validate routinely in the Java 8 deobfuscated Forge runner. Validate the packaged release in a representative modern-Java server before release, because the current GTNH deobfuscated `runServer17`/`runServer21`/`runServer25` tasks do not provide a working dedicated-server path in this checkout.
- Avoid modern runtime APIs whose downgrader stubs are unavailable or whose semantics materially differ on Java 8, especially in compatibility-critical and hot code.
## Audit findings
### Codebase size
@@ -200,11 +210,11 @@ If a characterization test captures a confirmed bug that the port intentionally
### Phase 0 — Freeze and measure the reference behavior
- [x] Produce and retain a reference dev jar from `f10595d` or the selected migration base.
- [ ] Dump the public/protected JVM API and important generated tile class shapes.
- [x] Dump the public/protected JVM API and important generated tile class shapes.
- [ ] Inventory downstream mods that compile against or reflect into ForgeMultipart.
- [ ] Record part IDs, NBT layouts, packet layouts, registration order, and side/lifecycle behavior.
- [x] Add a Java-8-compatible JUnit/Jupiter setup to the existing Gradle `test` task.
- [ ] Add a separate Forge integration-test source set/runner for behavior that cannot execute in a plain JVM.
- [x] Add a separate Forge integration-test source set/runner for behavior that cannot execute in a plain JVM.
- [x] Write the first characterization suite against the untouched Scala implementation.
- [ ] Create a remaining manual compatibility checklist for rendering, input, and other behavior that cannot be asserted reliably by the automated harness.
- [ ] Capture representative CPU and allocation profiles before optimization work.
@@ -403,3 +413,7 @@ These ranges were formed before designing the characterization suite and should
- Adopted characterization tests against the existing Scala implementation as a prerequisite for each converted area, backed by golden ABI/data fixtures and real Forge integration scenarios where plain JVM tests are insufficient.
- Bootstrapped JUnit Jupiter and passed an eight-case `IDWriter` characterization pilot against the untouched Scala implementation, covering byte/short/int thresholds and exact encoded bytes.
- The pilot found that negative `IDWriter.setMax` values select the byte carrier because Scala 2.11 treats `0xffffffff` as signed `-1`; current registry callers supply non-negative collection sizes, so this is recorded as a dormant edge case and possible deliberate divergence rather than changed during characterization.
- Added a dedicated `functionalTest` source set and `runFunctionalTestServer` task. The task starts a real Java 8 Forge dedicated server, loads a test-only FML mod, runs JUnit after `FMLServerStartedEvent`, writes legacy JUnit XML, and shuts the server down.
- Passed the first two Forge integration checks against the untouched Scala implementation: complete server lifecycle/world startup with all three ForgeMultipart mod IDs loaded, and generation plus constructor-class caching of a `TSlottedTile` composite with its 27-slot map.
- Hardened the Gradle wrapper to delete stale reports and fail when no fresh report is produced, no tests run, or any failure/error is reported. This is necessary because Forge 1.7.10 can terminate after a test-mod exception with process exit code 0.
- Confirmed the Java 8 server lane on Azul OpenJDK 8u492. The current deobfuscated modern-server tasks are not usable as release evidence: Java 21 needs the legacy Security Manager switch and then `Lwjgl3ifyRelauncherTweaker` rejects the server launch; Java 25 rejects `System.setSecurityManager` before Forge loads. Modern-Java validation therefore remains a packaged-server gate rather than a custom test-runner workaround.
+83 -1
View File
@@ -1 +1,83 @@
apply plugin: 'scala'
import org.gradle.api.tasks.GradleBuild
import groovy.xml.XmlSlurper
apply plugin: 'scala'
SourceSet functionalTestSet
sourceSets {
functionalTestSet = create("functionalTest") {
java {
srcDir("src/functionalTest/java")
compileClasspath += sourceSets.patchedMc.output + sourceSets.main.output
}
}
}
configurations {
named(functionalTestSet.compileClasspathConfigurationName).configure {
extendsFrom(named("compileClasspath").get())
}
named(functionalTestSet.runtimeClasspathConfigurationName).configure {
extendsFrom(named("runtimeClasspath").get())
}
named(functionalTestSet.annotationProcessorConfigurationName).configure {
extendsFrom(named("annotationProcessor").get())
}
}
tasks.register(functionalTestSet.jarTaskName, Jar) {
from(functionalTestSet.output)
archiveClassifier.set("functionalTests")
archiveVersion.set("1.0")
destinationDirectory.set(layout.buildDirectory.dir("tmp"))
}
def functionalTestRunProperty = "forgemultipart.withFunctionalTests"
def withFunctionalTests = providers.gradleProperty(functionalTestRunProperty).map { it.toBoolean() }.orElse(false).get()
def functionalTestReportDirectory = layout.projectDirectory.dir("run/server/junit-out")
def addFunctionalTestsToRun = { JavaExec task ->
task.dependsOn(functionalTestSet.jarTaskName)
task.classpath(
configurations.named(functionalTestSet.runtimeClasspathConfigurationName),
tasks.named(functionalTestSet.jarTaskName))
}
tasks.register("runFunctionalTestServer", GradleBuild) {
group = "verification"
description = "Runs the Java 8 Forge server with the ForgeMultipart functional test mod"
setTasks(["runServer"])
startParameter.projectProperties.put(functionalTestRunProperty, "true")
doFirst {
delete(functionalTestReportDirectory)
}
doLast {
def reports = fileTree(functionalTestReportDirectory).matching { include("TEST-*.xml") }.files
if (reports.isEmpty()) {
throw new GradleException("ForgeMultipart functional tests produced no JUnit XML report")
}
int tests = 0
int failures = 0
int errors = 0
reports.each { report ->
def suite = new XmlSlurper(false, false).parse(report)
tests += suite.@tests.toInteger()
failures += suite.@failures.toInteger()
errors += suite.@errors.toInteger()
}
if (tests == 0 || failures > 0 || errors > 0) {
throw new GradleException(
"ForgeMultipart functional tests failed: ${tests} tests, ${failures} failures, ${errors} errors")
}
}
}
if (withFunctionalTests) {
tasks.withType(JavaExec).configureEach { task ->
if (task.name == "runServer") {
addFunctionalTestsToRun(task)
}
}
}
+6
View File
@@ -8,6 +8,12 @@ dependencies {
testRuntimeOnly("org.junit.platform:junit-platform-launcher")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine")
functionalTestImplementation(platform("org.junit:junit-bom:5.11.4"))
functionalTestImplementation("org.junit.jupiter:junit-jupiter")
functionalTestImplementation("org.junit.platform:junit-platform-engine")
functionalTestImplementation("org.junit.platform:junit-platform-launcher")
functionalTestImplementation("org.junit.platform:junit-platform-reporting")
implementation("com.github.GTNewHorizons:CodeChickenCore:1.4.12:dev")
compileOnly("com.github.GTNewHorizons:Angelica:2.1.19:dev")
compileOnly("com.github.GTNewHorizons:Postea:1.2.5:dev") {transitive = false}
@@ -0,0 +1,60 @@
package codechicken.multipart.test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNotSame;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.BitSet;
import net.minecraft.server.MinecraftServer;
import org.junit.jupiter.api.Test;
import codechicken.multipart.TileMultipart;
import codechicken.multipart.asm.MultipartMixinFactory;
import codechicken.multipart.scalatraits.TSlottedTile;
import cpw.mods.fml.common.FMLCommonHandler;
import cpw.mods.fml.common.Loader;
import scala.collection.immutable.Nil$;
class ForgeEnvironmentSmokeTest {
@Test
void reachesServerStartedWithWorldAndMultipartModsLoaded() {
assertTrue(ForgeMultipartFunctionalTestMod.preInitialized);
assertTrue(ForgeMultipartFunctionalTestMod.initialized);
assertTrue(ForgeMultipartFunctionalTestMod.postInitialized);
assertTrue(ForgeMultipartFunctionalTestMod.serverAboutToStart);
assertTrue(ForgeMultipartFunctionalTestMod.serverStarted);
assertTrue(FMLCommonHandler.instance().getSide().isServer());
assertTrue(Loader.isModLoaded("ForgeMultipart"));
assertTrue(Loader.isModLoaded("ForgeMicroblock"));
assertTrue(Loader.isModLoaded("McMultipart"));
MinecraftServer server = MinecraftServer.getServer();
assertNotNull(server);
assertNotNull(server.worldServers);
assertTrue(server.worldServers.length > 0);
assertNotNull(server.worldServers[0]);
}
@Test
void generatesAndCachesSlottedTileClass() {
int traitId = MultipartMixinFactory.getId(TSlottedTile.class.getName().replace('.', '/'));
assertFalse(traitId < 0);
BitSet traits = new BitSet();
traits.set(traitId);
Object first = MultipartMixinFactory.construct(traits, Nil$.MODULE$);
Object second = MultipartMixinFactory.construct(traits, Nil$.MODULE$);
assertTrue(first instanceof TileMultipart);
assertTrue(first instanceof TSlottedTile);
assertNotSame(first, second);
assertSame(first.getClass(), second.getClass());
assertEquals(27, ((TSlottedTile) first).v_partMap().length);
}
}
@@ -0,0 +1,113 @@
package codechicken.multipart.test;
import java.io.File;
import java.io.PrintWriter;
import java.nio.file.FileSystems;
import java.nio.file.Path;
import net.minecraft.server.MinecraftServer;
import net.minecraft.util.ChatComponentText;
import org.junit.platform.engine.discovery.DiscoverySelectors;
import org.junit.platform.launcher.Launcher;
import org.junit.platform.launcher.LauncherDiscoveryRequest;
import org.junit.platform.launcher.LauncherSession;
import org.junit.platform.launcher.TestPlan;
import org.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder;
import org.junit.platform.launcher.core.LauncherFactory;
import org.junit.platform.launcher.listeners.SummaryGeneratingListener;
import org.junit.platform.launcher.listeners.TestExecutionSummary;
import org.junit.platform.reporting.legacy.xml.LegacyXmlReportGeneratingListener;
import cpw.mods.fml.common.Mod;
import cpw.mods.fml.common.event.FMLInitializationEvent;
import cpw.mods.fml.common.event.FMLPostInitializationEvent;
import cpw.mods.fml.common.event.FMLPreInitializationEvent;
import cpw.mods.fml.common.event.FMLServerAboutToStartEvent;
import cpw.mods.fml.common.event.FMLServerStartedEvent;
@Mod(
modid = "ForgeMultipartTests",
name = "ForgeMultipart Functional Tests",
version = "1.0",
dependencies = "required-after:ForgeMultipart",
acceptableRemoteVersions = "*")
public final class ForgeMultipartFunctionalTestMod {
static boolean preInitialized;
static boolean initialized;
static boolean postInitialized;
static boolean serverAboutToStart;
static boolean serverStarted;
@Mod.EventHandler
public void preInit(FMLPreInitializationEvent event) {
preInitialized = true;
}
@Mod.EventHandler
public void init(FMLInitializationEvent event) {
initialized = true;
}
@Mod.EventHandler
public void postInit(FMLPostInitializationEvent event) {
postInitialized = true;
}
@Mod.EventHandler
public void serverAboutToStart(FMLServerAboutToStartEvent event) {
serverAboutToStart = true;
}
@Mod.EventHandler
public void serverStarted(FMLServerStartedEvent event) {
serverStarted = true;
MinecraftServer server = MinecraftServer.getServer();
server.addChatMessage(new ChatComponentText("Running ForgeMultipart functional tests..."));
TestExecutionSummary summary = runTests();
if (summary.getTestsFoundCount() == 0) {
throw new IllegalStateException("No ForgeMultipart functional tests were discovered");
}
if (summary.getTotalFailureCount() > 0) {
throw new IllegalStateException("ForgeMultipart functional tests failed; check junit-out and the server log");
}
server.addChatMessage(new ChatComponentText("ForgeMultipart functional tests passed"));
server.initiateShutdown();
}
private static TestExecutionSummary runTests() {
System.setProperty("junit.platform.reporting.open.xml.enabled", "false");
Path reportDirectory = FileSystems.getDefault().getPath("./junit-out/").toAbsolutePath();
File reportDirectoryFile = reportDirectory.toFile();
reportDirectoryFile.mkdirs();
File[] oldReports = reportDirectoryFile.listFiles((directory, name) -> name.endsWith(".xml"));
if (oldReports != null) {
for (File oldReport : oldReports) {
oldReport.delete();
}
}
LauncherDiscoveryRequest discovery = LauncherDiscoveryRequestBuilder.request()
.selectors(DiscoverySelectors.selectPackage("codechicken.multipart.test"))
.build();
SummaryGeneratingListener summaryListener = new SummaryGeneratingListener();
PrintWriter stderr = new PrintWriter(System.err, true);
LegacyXmlReportGeneratingListener xmlListener = new LegacyXmlReportGeneratingListener(reportDirectory, stderr);
try (LauncherSession session = LauncherFactory.openSession()) {
Launcher launcher = session.getLauncher();
TestPlan plan = launcher.discover(discovery);
launcher.registerTestExecutionListeners(summaryListener, xmlListener);
launcher.execute(plan);
}
TestExecutionSummary summary = summaryListener.getSummary();
summary.printFailuresTo(stderr, 32);
summary.printTo(stderr);
stderr.flush();
return summary;
}
}