From 8d0ba7c76e91dd2d08eac995f781a7b1c91cbce7 Mon Sep 17 00:00:00 2001 From: Algent Date: Thu, 27 Aug 2026 13:52:06 +0300 Subject: [PATCH] docs: revise migration plan from consumer audit --- JAVA_MIGRATION.md | 117 ++++++++++++++++++++++--------- JAVA_MIGRATION_CONSUMER_AUDIT.md | 4 ++ JAVA_MIGRATION_DIVERGENCES.md | 19 ++++- JAVA_MIGRATION_HANDOFF.md | 17 ++++- JAVA_MIGRATION_MANUAL_CHECKS.md | 6 ++ 5 files changed, 123 insertions(+), 40 deletions(-) diff --git a/JAVA_MIGRATION.md b/JAVA_MIGRATION.md index d3bf81f..ae50a8a 100644 --- a/JAVA_MIGRATION.md +++ b/JAVA_MIGRATION.md @@ -23,15 +23,24 @@ No trustworthy tool will produce a maintainable Java port automatically. A decom ## Current status -Phase 0 is complete apart from broad NBT/packet fixtures and performance baselines. Phase 3 is in progress and is -where the work currently sits. Phase 1 has not properly started: GTNHLib is present but was added at `api` scope to -satisfy a test compile rather than because a migration change needed it, and no Forge mod dependency was declared, so -that entry should be revisited rather than treated as done. Phase 8 is answered and deferred. +The binary and source-level consumer audits are complete. Phase 0 still lacks broad NBT/packet fixtures and a +pre-optimization profile. The low-coupling Phase 3 queue is complete; before entering measured hot-path or generated +tile work, the immediate milestone is the audit-derived compatibility gate below. + +That audit found one existing regression on this branch: Schematica 1.12.6 reflects the original private +`MultiPartRegistry$` field `codechicken$multipart$MultiPartRegistry$$typeMap` and casts it to a Scala mutable map. The +Java registry port moved the state to a private Java `HashMap` on `MultiPartRegistry`, so Schematica's integration +currently disables itself. Restoring and testing that compatibility view is the next task. + +Phase 1 has not properly started: GTNHLib is present but was added at `api` scope to satisfy a test compile rather +than because a migration change needed it, and no Forge mod dependency was declared. Revisit that entry only when a +real migration change needs it. Phase 8 is answered and deferred. The working rules established while converting, which apply to every remaining area: -- Check `JAVA_MIGRATION_ABI_INVENTORY.md` before writing a compatibility bridge. Several were written speculatively and - later deleted; the inventory is the authority on what is load-bearing. +- Check both consumer audits before changing a compatibility surface. `JAVA_MIGRATION_ABI_INVENTORY.md` is the + authority on binary linkage; `JAVA_MIGRATION_CONSUMER_AUDIT.md` is the authority on runtime behavior, data formats, + reflection, and mixin field access. - Emit a Java `default` method only when no class in an implementor's superclass chain declares the same member. Otherwise leave it abstract and add explicit forwarders, because a superclass method always beats an interface default and the failure is silent. @@ -230,8 +239,10 @@ If a characterization test captures a confirmed bug that the port intentionally - [x] Produce and retain a reference dev jar from `f10595d` or the selected migration base. - [x] Dump the public/protected JVM API and important generated tile class shapes. -- [x] Inventory downstream mods that compile against or reflect into ForgeMultipart. See `JAVA_MIGRATION_ABI_INVENTORY.md`. -- [ ] Record part IDs, NBT layouts, packet layouts, registration order, and side/lifecycle behavior. +- [x] Inventory downstream mods that compile against or reflect into ForgeMultipart. See + `JAVA_MIGRATION_ABI_INVENTORY.md` and `JAVA_MIGRATION_CONSUMER_AUDIT.md`. +- [x] Record source-level consumers of part IDs, NBT, packets, registration order, reflection, and lifecycle behavior. +- [ ] Freeze representative part IDs, NBT trees, packet bytes, registration order, and lifecycle behavior as tests. - [x] Add a Java-8-compatible JUnit/Jupiter setup to the existing Gradle `test` task. - [x] Add a separate Forge integration-test source set/runner for behavior that cannot execute in a plain JVM. - [x] Write the first characterization suite against the untouched Scala implementation. @@ -270,22 +281,37 @@ Exit condition: new code can use a clean Java API while existing supported binar ### Phase 3 — Convert low-coupling code - [x] Characterize each related leaf group before converting it. -- [ ] Convert constants, value holders, simple utilities, registries, packet data helpers, and leaf classes first. -- [ ] Replace `JavaConversions` with explicit Java collections or controlled adapters. +- [x] Convert the identified constants, value holders, simple utilities, registries, packet data helpers, and leaf + classes. +- [x] Replace `JavaConversions` in converted files with explicit Java collections or controlled adapters. - [x] Preserve iteration order and null behavior. - [x] Keep mixed Scala/Java compilation working throughout the phase. -Status: this is the phase currently in progress. Converted so far, each with characterization committed separately -first and a frozen reference-compiled consumer where a bridge is load-bearing: `IDWriter`, `PartialOcclusionTest` and -`JPartialOcclusion`, `TCuboidPart` and `JCuboidPart`, the `TNormalOcclusion` unit including `NormalOcclusionTest` and -`NormallyOccludedPart`, `TFacePart`, the `TIconHitEffects` unit including `IconHitEffects` and `JIconHitEffects`, -`TItemMultiPart` and `JItemMultiPart`, `TEdgePart`, `Saw`, and `MicroMaterialRegistry` with its two nested interfaces. - -All eight load-bearing `$class` helpers identified by the ABI inventory are now Java with their bridges retained. -`JavaConversions` is removed from every converted file; the remaining uses are in files not yet converted. +Status: **complete.** The low-risk queue is empty. All eight load-bearing `$class` helpers are Java with their bridges +retained, as are both registries, the central part/tile types, scheduler/helper utilities, `BlockMultipart`, and +`MultipartRenderer`. The exact completed list is maintained in `JAVA_MIGRATION_HANDOFF.md`. Exit condition: leaf code is Java and no longer generates avoidable Scala closures or collection adapters. +### Immediate compatibility gate — current next milestone + +Do this before another source conversion: + +- [ ] Restore Schematica's reflective `MultiPartRegistry$` type-map view with its exact field name and Scala mutable-map + shape, backed by the canonical Java registry state; add a regression test that performs Schematica's actual lookup. +- [ ] Add exact member-level guards for GuideNH's `BlockMicroMaterial.block/meta` mixin fields, Et Futurum's mutable + `ButtonPart.metaSideMap/sideMetaMap`, Iguana's `ItemSaw.harvestLevel`, and Galacticraft's name-only + `registerMaterial` lookup. +- [ ] Freeze `TileMultipart` order and lifecycle observations: `parts`/`id` NBT, `partList`/`jPartList`, `partMap`, slot + rebinding, add/remove/replace callback order, moving the live tile, and `onMoved`. +- [ ] Add compact mixed-part NBT and description-packet fixtures. Use representative fake parts in the existing test + harness; do not add all downstream mods as test dependencies. +- [ ] Freeze one generated Scala trait, one Java trait, and representative pass-through interfaces before changing + built-in tile traits or either generator. + +Exit condition: every source-only constraint that can silently fail has an automated structural/behavioral guard, +and the already-ported registry again supports Schematica. + ### Phase 4 — Convert measured hot paths - [ ] Characterize result ordering, callback timing, early exits, and edge cases before changing traversal code. @@ -295,6 +321,11 @@ Exit condition: leaf code is Java and no longer generates avoidable Scala closur - [ ] Use fastutil only where profiling and data shape justify it. - [ ] Re-profile the same scenarios and record both improvements and regressions. +Status: not started. This phase does not begin until the immediate compatibility gate and an initial profile are +complete. The first sensible implementation unit is the whole `IRedstonePart.scala` file: its six related traits and +load-bearing `RedstoneInteractions$` must be characterized and ported together. Do not optimize `TileMultipart` +storage merely because the source audit found allocations; its list/order/lifecycle contracts must be frozen first. + Exit condition: identified steady-state Scala allocation sites are removed with compatible results. ### Phase 5 — Convert built-in tile traits through the existing Java path @@ -305,6 +336,8 @@ Exit condition: identified steady-state Scala allocation sites are removed with - [ ] Convert remaining traits in small related groups. - [ ] Document or deliberately relax the Java trait restrictions only when a real trait requires it. - [ ] Preserve pass-through interface behavior. +- [ ] For the pilot, explicitly cover OpenComputers' `TSlottedTile.v_partMap` rebinding, AE2's + `TIInventoryTile.rebuildSlotMap`, and ProjectRed/Extra Utilities `TRedstoneTile` behavior. Exit condition: built-in behavior is implemented in Java while the established runtime composition mechanism remains stable. @@ -324,6 +357,8 @@ Exit condition: normal runtime implementation is Java; any remaining Scala is is - [ ] Translate the existing compiler/generator into readable Java while preserving emitted class shapes and cache keys. - [ ] Evaluate GTNHLib ASM helpers where they reduce boilerplate without altering semantics. - [ ] Keep Scala-trait registration while deprecated bridges or supported external Scala traits require it. +- [ ] Run ProjectRed's `LightMicroblock` external Scala-trait fixture and pass-through-interface fixtures against the + ported generator. - [ ] Decide separately whether a future generator design can avoid custom bytecode generation. Do not couple that experiment to the source-language port. Exit condition: the composition subsystem is maintainable Java with behavior and generated ABI verified against the reference. @@ -357,7 +392,10 @@ The work should move from low to high risk while keeping a buildable mixed-langu Extensive tests are both possible and recommended here. Their purpose is not to prove that the current Scala behavior is ideal; it is to make its observable behavior explicit before translation, so every later difference is intentional. -The repository currently has no test sources or test dependencies. The GTNH Gradle convention and Scala plugin provide a normal Gradle test path, so no custom framework is needed for deterministic logic. Minecraft 1.7.10 does not provide the modern GameTest framework, therefore game-dependent coverage needs a small Forge integration harness rather than pretending all Minecraft behavior can run in an ordinary unit test. +The repository now has a Java-8-compatible JUnit Jupiter suite and a small Forge integration runner. The current +baseline is 125 plain-JVM tests and 22 Forge server tests. Minecraft 1.7.10 does not provide the modern GameTest +framework, so game-dependent coverage continues through the existing narrow Forge harness rather than a second test +framework. ### Layer 1 — Plain JVM characterization tests @@ -415,27 +453,38 @@ Coverage percentage is not the goal. The required evidence for an area is: - The tests demonstrably pass against the old Scala area before any Java replacement begins. - The same tests pass unchanged against the Java implementation, except for separately documented bug fixes. -## Open decisions +## Decisions -These decisions should be made from ecosystem evidence rather than assumed during translation: +Resolved from the consumer audits: -1. Which existing downstream binaries must keep working without recompilation? -2. Are third-party Scala traits registered through `registerScalaTrait`, or is that path only used internally? -3. How long should deprecated Scala bridges remain supported? -4. Is complete Scala runtime removal required for the first Java release, or is Java-maintainable source the first milestone? -5. Which known bugs are intentionally fixed during the port, and which are preserved until compatibility is established? -6. Which GTNHLib release is the minimum supported version for the target pack? -7. After the Java port is stable, is there enough benefit to redesign runtime tile composition, or should the Java ASM generator remain? +1. All 27 binaries in the target pack must keep working without recompilation; UtilitiesInExcess is an additional + forward-compatibility target. +2. Third-party Scala trait registration is public in practice: ProjectRed registers `LightMicroblock`. +3. Java-maintainable source is the first-release milestone. Scala runtime removal is deferred because shipping mods + link Scala descriptors, trait helpers, companions, and external trait generation. -## Rough effort expectation +Still open, and only to be decided when they become necessary: -These are order-of-magnitude estimates for one developer familiar with Forge 1.7.10, excluding long ecosystem validation: +1. How long deprecated Scala bridges remain supported after the initial Java release. +2. Which known bugs are intentionally fixed during the port rather than after compatibility is established. +3. Which GTNHLib release is the minimum supported version once code actually needs it. +4. Whether runtime tile composition merits redesign after the Java port is stable. -- Targeted performance cleanup while retaining Scala: roughly 3–7 focused days. -- Java implementation while retaining deprecated Scala ABI bridges and the current composition model: roughly 4–8 weeks. -- Full Java implementation plus safe Scala runtime removal and broader compatibility work: roughly 8–14+ weeks. +## Effort tracking -These ranges were formed before designing the characterization suite and should now be treated as lower bounds. The tests add deliberate work before each conversion but should replace a substantial amount of later regression debugging. Revise the estimates after the first low-coupling subsystem completes the full characterize-port-compare cycle and after Phase 0 determines how much of the Scala ABI and Scala trait mechanism is used externally. +The old pre-audit calendar ranges are retired. They mixed the Java source port, ecosystem validation, performance +work, and optional Scala-runtime removal into estimates that are no longer useful. Track the remaining work by +completed compatibility gates instead: + +1. Repair the Schematica registry view and finish the source-only reflection/member guards. +2. Freeze tile ordering, lifecycle, NBT, packet, and generator behavior with compact representative fixtures. +3. Capture the first focused CPU/allocation baseline and port the complete redstone unit. +4. Convert a representative generated tile-trait group, then the remaining trait groups. +5. Convert the multipart/microblock core while keeping the frozen data and consumer contracts green. +6. Update the ASM/runtime generator, then run packaged-pack and manual release validation. + +Estimate calendar time again after gates 1 and 2. They expose the real cost of hidden consumer contracts and the +generated-trait compatibility work. Scala-runtime removal remains a later project and is not part of this estimate. ## Findings log diff --git a/JAVA_MIGRATION_CONSUMER_AUDIT.md b/JAVA_MIGRATION_CONSUMER_AUDIT.md index 0fb9c7b..708e785 100644 --- a/JAVA_MIGRATION_CONSUMER_AUDIT.md +++ b/JAVA_MIGRATION_CONSUMER_AUDIT.md @@ -351,6 +351,10 @@ effective public API**. A Java `Map` replacement will silently disable Schematic is retained or Schematica is patched and deployed first. This is a higher-risk constraint than the constant-pool inventory alone shows. +**Current branch:** this is already broken by the Java `MultiPartRegistry` port. The canonical state is now a private +Java `HashMap` on `MultiPartRegistry`, while `MultiPartRegistry$` has no field with the reflected name. Restoring and +testing a compatibility view is the immediate migration blocker. + ### UtilitiesInExcess — future replacement, already a substantial consumer UtilitiesInExcess is not in the `+700` JAR scan but is in scope because it is intended to replace Extra Utilities. diff --git a/JAVA_MIGRATION_DIVERGENCES.md b/JAVA_MIGRATION_DIVERGENCES.md index c121e4c..c3777ec 100644 --- a/JAVA_MIGRATION_DIVERGENCES.md +++ b/JAVA_MIGRATION_DIVERGENCES.md @@ -567,12 +567,23 @@ Schematica reaches the singleton reflectively. ### Observable behavior -No known divergence. `beforeServerStart` still sorts the type map by name so ids are indexes into it, part ids still +Registry behavior itself has no known divergence. `beforeServerStart` still sorts the type map by name so ids are indexes into it, part ids still round trip through the shared `IDWriter`, converters are still grouped per block with the first non-null result winning, registration still refuses to run once the registry is closed and still requires an active mod container, a duplicate part id still throws, and `writePartID` and `getModContainer` still throw rather than returning null for an unknown name. +### Known reflective regression, discovered 2026-08-27 + +The full consumer-source audit found a constraint that the bytecode scan missed. Schematica 1.12.6 obtains the private +field `codechicken$multipart$MultiPartRegistry$$typeMap` from `MultiPartRegistry$` and casts its value to +`scala.collection.mutable.Map`. The Java port moved the canonical map to a private `java.util.HashMap` on +`MultiPartRegistry`, and `MultiPartRegistry$` exposes no compatibility field, so Schematica currently disables its FMP +integration during reflective initialization. + +This is not an accepted divergence. Restore an exact reflective compatibility view backed by the canonical Java +registry state and add a test reproducing Schematica's field lookup before continuing the migration. + ### Preserved Scala-typed descriptors Both remain, because ForgeRelocationFMP and ProjRed link against them: @@ -595,8 +606,10 @@ resolves against a plain array parameter. It also makes the Java call sites more - `readIDMap` returns `java.util.List` instead of `scala.collection.Seq`, as with `MicroMaterialRegistry`. It is `private[multipart]` in Scala, so it has no static forwarder, and no jar references it. -- The mangled public accessors Scala emitted so its closures could reach private state - (`codechicken$multipart$MultiPartRegistry$$typeMap()` and the four others) are gone. No jar references them. +- The mangled public accessor methods Scala emitted so its closures could reach private state + (`codechicken$multipart$MultiPartRegistry$$typeMap()` and the four others) are gone. No jar links those methods. + This original check did not cover Schematica's reflective access to the separate private backing field described + above. ### Compiler artifacts diff --git a/JAVA_MIGRATION_HANDOFF.md b/JAVA_MIGRATION_HANDOFF.md index c56566d..09aa186 100644 --- a/JAVA_MIGRATION_HANDOFF.md +++ b/JAVA_MIGRATION_HANDOFF.md @@ -7,6 +7,7 @@ what breaks, and what is left. The other documents hold the reasoning. | --- | --- | | `JAVA_MIGRATION.md` | The plan, phase state, and a running findings log | | `JAVA_MIGRATION_ABI_INVENTORY.md` | Which downstream mods use what. **The authority on whether anything is load-bearing** | +| `JAVA_MIGRATION_CONSUMER_AUDIT.md` | How every consumer uses FMP at runtime: data, lifecycle, reflection, and generated tiles | | `JAVA_MIGRATION_DIVERGENCES.md` | Every intentional difference from the reference, one entry per port | | `JAVA_MIGRATION_MANUAL_CHECKS.md` | What no automated test can cover, and must be checked by hand in a client | @@ -14,8 +15,8 @@ Branch: `algent/java`. Base: `master`. 54 commits so far. ## The one rule that matters -**Check `JAVA_MIGRATION_ABI_INVENTORY.md` before writing a compatibility bridge.** Several were written speculatively -early on and later deleted. The inventory is generated from real bytecode, not guesswork, and it is the authority. +**Check both consumer audits before changing a compatibility surface.** The ABI inventory is the authority on what +must link; the source-level audit is the authority on behavior, serialized data, reflection, and mixin field access. ## Workflow per type @@ -191,8 +192,18 @@ Both `package.scala` objects are gone, removed rather than ported. `MultipartRen **The low-risk queue is empty.** Pick the next piece deliberately rather than off the top of a list. +**Immediate blocker: repair the already-ported registry for Schematica.** Schematica 1.12.6 reflects +`MultiPartRegistry$.codechicken$multipart$MultiPartRegistry$$typeMap` and casts it to a Scala mutable map. The current +Java port has only a private Java `HashMap` on `MultiPartRegistry`, so that integration is broken. Restore an exact +compatibility view backed by the canonical Java state and freeze Schematica's lookup in a test before any new port. + +Then close the small audit-derived test gap: exact reflection/mixin fields, tile list/order/move lifecycle, compact +mixed NBT/packet fixtures, and representative generated-trait/pass-through fixtures. The checklist is in the +"Immediate compatibility gate" section of `JAVA_MIGRATION.md`. + `IRedstonePart.scala` is misleadingly named and is **not** a marker-trait file. It holds six traits plus -`RedstoneInteractions`, whose `MODULE$` is load-bearing, so it is its own piece of work at medium risk. +`RedstoneInteractions`, whose `MODULE$` is load-bearing. After the compatibility gate and an initial profile, port the +whole file as one medium-risk unit; do not split its mutually dependent behavior into smaller commits. **Medium.** The `handler` packages on both sides, `MultipartRenderer`, `MicroRecipe`, `ItemMicroPart`, `MicroblockPlacement`, `PlacementGrids`, `BlockMicroMaterial`, `MissingMicroMaterial`, `ConfigContent`, diff --git a/JAVA_MIGRATION_MANUAL_CHECKS.md b/JAVA_MIGRATION_MANUAL_CHECKS.md index 14a8753..096a552 100644 --- a/JAVA_MIGRATION_MANUAL_CHECKS.md +++ b/JAVA_MIGRATION_MANUAL_CHECKS.md @@ -35,6 +35,12 @@ release that includes those ports. | --- | --- | --- | | Load the pack with ProjRed installed and place illumar lamp microblocks | The lamps generate, render and light correctly | `MicroblockGenerator.registerTrait` Scala trait path | | Place and break OpenComputers, ProjectBlue and AE2 parts | No `NoSuchMethodError` or `AbstractMethodError` in the log | retained `$class` bridges | +| Load and render a schematic containing an ordered mixed multipart tile | Every part appears in the preview in the saved order; the integration does not silently disable itself | Schematica private registry-map reflection and tile NBT reconstruction | +| Open a GuideNH scene containing a multipart tile and export its part/material data | The preview, material data and part statistics are complete | companion reflection, `partList_$eq`, and `BlockMicroMaterial` mixin fields | +| Place FMP buttons on every supported face with Et Futurum loaded | Button orientation matches the fixed vanilla behavior | reflected mutable `ButtonPart` side/meta arrays | +| Toggle an OpenComputers print that shares a block with another slotted part | Its slots are rebuilt without losing, hiding or disconnecting either part | external mutation of `TSlottedTile.v_partMap` plus `bindPart` | +| Move the same composite tile once through ForgeRelocationFMP and once through MatterManipulator | Coordinates, part-to-tile references, generated interfaces and rendering remain correct after `onMoved` | live multipart move lifecycle | +| Inspect a mixed multipart tile with Waila | Providers receive every saved part ID and show the expected part data | reflected block identity and tile NBT `parts`/`id` layout | | Load a world saved before the port | All microblocks and covers keep their material and shape | `MicroMaterialRegistry` id map | | Join a server whose material set differs from the client | The client is disconnected with the missing material list, not a crash | `readIDMap` |