mirror of
https://github.com/KAMKEEL/CustomNPC-Plus.git
synced 2026-11-20 09:59:42 +00:00
MORE ESSENTIAL MIGRATION TOOLS
This commit is contained in:
1 parent
f3f800e166
commit
e26f8a4a01
6 files changed
+47552
-14
No files matched your search
+148
-3
@@ -20,7 +20,8 @@
|
||||
|------|------|---------|------------|
|
||||
| **`pa_batch.py`** | **Python** | **BATCH create 5+ Java files from JSON manifest (auto-derives paths)** | **`manifest.json`, `--force`, `--dry-run`** |
|
||||
| **`pa_editor.py`** | **Python** | **BATCH edit existing files from JSON manifest (atomic insert/delete/replace)** | **`manifest.json`, `--force`, `--dry-run`, `--backup`** |
|
||||
| `Copy-Recursive` | PS | Copy files from mc1710 to core, recursively following imports | `-Sources`, `-AllowPaths` |
|
||||
| **`pa_analyzer.py`** | **Python** | **Analyze PA member usage across mc1710 from classification + index data** | **`--execute`, `--pa`, `--classification`, `--verbose`** |
|
||||
| `Copy-Recursive` | PS | Copy files from mc1710 to core, recursively following imports | `-Sources`, `-AllowPaths`, `-AllowRecursePaths` |
|
||||
| `Symbol-Swap` | PS | Replace type names + import surgery across all files | `-Directory`, `-Swaps` |
|
||||
| `Nuke-Imports` | PS | Remove all forbidden imports from a directory | `-Directory`, `-ForbiddenPrefixes` |
|
||||
| `Create-And-Propagate` | PS | Create a PA interface AND replace its MC equivalent everywhere atomically | `-FilePath`, `-Content`, `-TargetDirectory`, `-Swaps` |
|
||||
@@ -415,18 +416,45 @@ python tools/migration/test_pa_editor.py
|
||||
|-------|----------|---------|-------------|
|
||||
| `-Sources` | Yes | -- | Array of relative paths within McRoot (e.g., `'noppes/npcs/controllers/DialogController.java'`) |
|
||||
| `-AllowPaths` | No | `@()` (all allowed) | WHITELIST of path prefixes. Only files under these paths get recursively copied. Initial `-Sources` always bypass this. |
|
||||
| `-AllowRecursePaths` | No | `@()` (all recursed) | RECURSION WHITELIST: only follow/scan imports of files whose path starts with one of these prefixes. Files from non-whitelisted paths are still COPIED but their imports are NOT recursively followed. Initial `-Sources` always have their imports followed. **Different from `-AllowPaths`** — see below. |
|
||||
| `-McRoot` | No | `'mc1710/src/main/java'` | Source root |
|
||||
| `-CoreRoot` | No | `'core/src/main/java'` | Destination root |
|
||||
| `-ProjectRoot` | No | Auto-detected | Absolute project root |
|
||||
| `-ForbiddenPrefixes` | No | java/MC/Forge/etc | Import prefixes never followed |
|
||||
|
||||
**`-AllowPaths` vs `-AllowRecursePaths` — Critical Distinction:**
|
||||
```
|
||||
-AllowPaths = WHERE files can be copied TO (destination filter)
|
||||
-AllowRecursePaths = WHERE imports can be recursively FOLLOWED FROM (recursion depth filter)
|
||||
```
|
||||
|
||||
| Scenario | `-AllowPaths` | `-AllowRecursePaths` | Result |
|
||||
|----------|---------------|---------------------|--------|
|
||||
| File in controllers/ imports file in util/ | util/ in AllowPaths | util/ NOT in AllowRecursePaths | util/ file IS copied but its imports are NOT followed |
|
||||
| File in controllers/ imports file in roles/ | roles/ NOT in AllowPaths | N/A | roles/ file is NOT copied at all |
|
||||
| File in controllers/ imports file in config/ | config/ in AllowPaths | config/ in AllowRecursePaths | config/ file IS copied AND its imports ARE followed |
|
||||
|
||||
**Behavior:**
|
||||
- Initial `-Sources` files are ALWAYS copied regardless of whitelist.
|
||||
- Files already in core are NOT re-copied but ARE scanned for transitive deps.
|
||||
- Initial `-Sources` files ALWAYS have their imports followed regardless of `-AllowRecursePaths`.
|
||||
- Files already in core are NOT re-copied but ARE scanned for transitive deps (if recursion allowed).
|
||||
- Handles wildcard imports (`import foo.bar.*`), static imports, inner class imports.
|
||||
- Returns array of copied file paths.
|
||||
|
||||
**Output summary shows:** COPIED, EXISTS, skipped(not whitelisted), not found.
|
||||
**Output summary shows:** COPIED, EXISTS, skipped(not whitelisted), recursion restricted, not found.
|
||||
|
||||
**Summary line format:** `Copied: X files (Y initial sources, Z recursively discovered, A non-recurse imports)`
|
||||
|
||||
**Usage example with `-AllowRecursePaths`:**
|
||||
```powershell
|
||||
# Copy DialogController, allow copying from controllers/ and util/,
|
||||
# but only recurse into imports from controllers/.
|
||||
# Files from util/ will be copied as dependencies but their imports won't be followed.
|
||||
Copy-Recursive `
|
||||
-Sources @('noppes/npcs/controllers/DialogController.java') `
|
||||
-AllowPaths @('noppes/npcs/controllers/', 'noppes/npcs/util/') `
|
||||
-AllowRecursePaths @('noppes/npcs/controllers/')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -1232,6 +1260,123 @@ powershell -ExecutionPolicy Bypass -File "tools/migration/test_runner.ps1"
|
||||
---
|
||||
---
|
||||
|
||||
# PA Analyzer — `pa_analyzer.py`
|
||||
|
||||
> **Maps which MC type members each PA uses across mc1710 source files.**
|
||||
>
|
||||
> Reads `classification_results.json` (non-suppressed PAs) and `mc_usage_index.json` (member usage) to produce a deterministic JSON mapping: `{ "PAName": { mc_types, members, source_files, usage_count, confidence } }`.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# Dry-run (default) — shows summary without writing
|
||||
python tools/migration/pa_analyzer.py
|
||||
|
||||
# Execute — writes output JSON
|
||||
python tools/migration/pa_analyzer.py --execute
|
||||
|
||||
# Filter to specific PAs
|
||||
python tools/migration/pa_analyzer.py --execute --pa INbt --pa IPlayer
|
||||
|
||||
# Filter by classification
|
||||
python tools/migration/pa_analyzer.py --execute --classification EXISTING INTERFACE
|
||||
|
||||
# Verbose (detailed per-PA breakdown)
|
||||
python tools/migration/pa_analyzer.py --execute --verbose
|
||||
|
||||
# Machine-readable JSON summary to stdout
|
||||
python tools/migration/pa_analyzer.py --execute --json
|
||||
|
||||
# Custom output path
|
||||
python tools/migration/pa_analyzer.py --execute --output /tmp/pa_usage.json
|
||||
```
|
||||
|
||||
## CLI Options
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `--execute` | false | Actually write output JSON (default: dry-run preview) |
|
||||
| `--output` | `tools/migration/pa_usage_analysis.json` | Output path |
|
||||
| `--classifications` | `tools/migration/classification_results.json` | Input classifications |
|
||||
| `--index` | `tools/migration/mc_usage_index.json` | Input usage index |
|
||||
| `--pa` | all | Filter to specific PA name(s). Repeatable. |
|
||||
| `--classification` | all non-suppressed | Filter by classification: EXISTING, INTERFACE, AUTO |
|
||||
| `--json` | false | Machine-readable JSON summary to stdout |
|
||||
| `--verbose` | false | Show detailed per-PA member breakdown |
|
||||
| `--project-root` | auto-detect | Project root override |
|
||||
|
||||
## Output Format
|
||||
|
||||
```json
|
||||
{
|
||||
"metadata": {
|
||||
"analysis_date": "2026-03-15T23:03:13Z",
|
||||
"tool_version": "1.0.0",
|
||||
"total_PAs_scanned": 238,
|
||||
"total_mc_types_covered": 245,
|
||||
"confidence_thresholds": { "high": ">=90%", "medium": ">=50%", "low": "<50%" }
|
||||
},
|
||||
"summary": {
|
||||
"by_confidence": { "high": ["INbt", ...], "medium": [...], "low": [...] },
|
||||
"by_classification": { "EXISTING": [...], "INTERFACE": [...], "AUTO": [...] },
|
||||
"top_by_usage": [{ "pa_name": "INbt", "usage_count": 4452, "file_count": 356 }, ...],
|
||||
"unused_pas": ["IEnchantment", ...]
|
||||
},
|
||||
"pa_usage": {
|
||||
"INbt": {
|
||||
"mc_types": [{ "fqn": "net.minecraft.nbt.NBTTagCompound", "shortName": "NBTTagCompound", "classification": "EXISTING", "import_count": 593 }],
|
||||
"classifications": ["EXISTING"],
|
||||
"members": {
|
||||
"methods": { "setInteger": { "count": 569, "file_count": 164, "files": [...] } },
|
||||
"static_methods": {},
|
||||
"fields": {}
|
||||
},
|
||||
"source_files": ["noppes/npcs/controllers/FactionController.java", ...],
|
||||
"usage_count": 4452,
|
||||
"unique_members": 32,
|
||||
"file_count": 356,
|
||||
"constructors": { "count": 0, "file_count": 0, "files": [] },
|
||||
"casts": { "count": 0, "file_count": 0, "files": [] },
|
||||
"instanceof": { "count": 5, "file_count": 4, "files": [...] },
|
||||
"confidence": 100.0
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Confidence Scoring
|
||||
|
||||
Weighted score from usage density, member coverage, and file spread:
|
||||
- **Usage density (40%)**: `min(total_usage / 100, 1.0)` — caps at 100 usages
|
||||
- **Member coverage (30%)**: `min(unique_members / 10, 1.0)` — caps at 10 unique members
|
||||
- **File spread (30%)**: `min(file_count / 20, 1.0)` — caps at 20 files
|
||||
|
||||
| Level | Threshold | Meaning |
|
||||
|-------|-----------|---------|
|
||||
| High | ≥90% | Core PA — heavily used, well-covered, wide spread |
|
||||
| Medium | ≥50% | Significant PA — meaningful usage worth abstracting |
|
||||
| Low | <50% | Niche PA — limited usage, may not need full interface |
|
||||
| Unused | 0% | No member usage found in index — review classification |
|
||||
|
||||
## Integration with Migration Workflow
|
||||
|
||||
```
|
||||
PHASE 0: PA ANALYSIS (run once, reference throughout)
|
||||
├── pa_analyzer.py --execute → full PA usage map
|
||||
├── Review unused_pas list → candidates for SUPPRESS
|
||||
├── Review high-confidence PAs → these MUST have complete interfaces
|
||||
└── Compare pa_usage members vs surface-miner → find gaps in existing interfaces
|
||||
|
||||
PHASE 1: Use pa_usage_analysis.json during migration
|
||||
├── Check PA entry before creating interface → know exactly which methods to include
|
||||
├── Cross-reference with surface-miner → pa_analyzer shows WHAT is used,
|
||||
│ surface-miner shows what's MISSING from the interface
|
||||
└── Review cast/instanceof counts → high counts = harder migration
|
||||
```
|
||||
|
||||
---
|
||||
---
|
||||
|
||||
# Diagnostic Toolchain — `analyze.py`
|
||||
|
||||
> **Diagnostic tools answer questions. Migration tools change code. ALWAYS diagnose first.**
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
# - Checks if the file's path starts with an ALLOWED prefix (whitelist)
|
||||
# - Checks if it already exists in core/
|
||||
# - If allowed AND not in core, copies it
|
||||
# - Then recursively processes THAT file's imports
|
||||
# - Then recursively processes THAT file's imports (subject to AllowRecursePaths)
|
||||
# 4. Skips all Minecraft/Forge/Java stdlib imports
|
||||
# 5. Logs every file it copies and every file it skips (with reason)
|
||||
#
|
||||
@@ -20,11 +20,24 @@
|
||||
# Copy-Recursive -Sources @('noppes/npcs/controllers/AnimationController.java') `
|
||||
# -AllowPaths @('noppes/npcs/controllers/', 'noppes/npcs/roles/', 'noppes/npcs/quests/', 'noppes/npcs/config/')
|
||||
#
|
||||
# # Copy with recursion restricted to controllers/ only:
|
||||
# Copy-Recursive -Sources @('noppes/npcs/controllers/AnimationController.java') `
|
||||
# -AllowPaths @('noppes/npcs/controllers/', 'noppes/npcs/roles/', 'noppes/npcs/quests/') `
|
||||
# -AllowRecursePaths @('noppes/npcs/controllers/')
|
||||
#
|
||||
# PARAMS:
|
||||
# -Sources Array of relative paths within McRoot (e.g., 'noppes/npcs/controllers/Foo.java')
|
||||
# -AllowPaths WHITELIST: only recursively copy files whose path starts with one of these prefixes.
|
||||
# The initial -Sources files are ALWAYS copied regardless of whitelist.
|
||||
# If empty/not provided, ALL project files are allowed (no filter).
|
||||
# -AllowRecursePaths RECURSION WHITELIST: only follow/scan imports of files whose path starts with
|
||||
# one of these prefixes. Files from non-whitelisted paths are still COPIED as
|
||||
# dependencies, but their own imports are NOT recursively followed.
|
||||
# Initial -Sources files ALWAYS have their imports followed regardless.
|
||||
# If empty/not provided, ALL files have their imports followed (current behavior).
|
||||
# DIFFERENT from -AllowPaths:
|
||||
# -AllowPaths = WHERE files can be copied TO (destination filter)
|
||||
# -AllowRecursePaths = WHERE imports can be recursively FOLLOWED FROM (recursion filter)
|
||||
# -McRoot Source root directory (default: 'mc1710/src/main/java')
|
||||
# -CoreRoot Destination root directory (default: 'core/src/main/java')
|
||||
# -ProjectRoot Absolute project root (auto-detected from script location if not provided)
|
||||
@@ -42,6 +55,8 @@ function Copy-Recursive {
|
||||
|
||||
[string[]]$AllowPaths = @(),
|
||||
|
||||
[string[]]$AllowRecursePaths = @(),
|
||||
|
||||
[string]$McRoot = 'mc1710/src/main/java',
|
||||
[string]$CoreRoot = 'core/src/main/java',
|
||||
[string]$ProjectRoot,
|
||||
@@ -66,10 +81,14 @@ function Copy-Recursive {
|
||||
$script:copied = [System.Collections.ArrayList]::new()
|
||||
$script:skippedExisting = [System.Collections.ArrayList]::new()
|
||||
$script:skippedNotAllowed = [System.Collections.ArrayList]::new()
|
||||
$script:skippedRecurse = [System.Collections.ArrayList]::new()
|
||||
$script:notFound = [System.Collections.ArrayList]::new()
|
||||
$script:initialSourceCount = 0
|
||||
$script:nonRecurseImportCount = 0
|
||||
$script:depth = 0
|
||||
|
||||
$hasWhitelist = $AllowPaths.Count -gt 0
|
||||
$hasRecurseWhitelist = $AllowRecursePaths.Count -gt 0
|
||||
|
||||
function Is-Forbidden {
|
||||
param([string]$Import)
|
||||
@@ -88,6 +107,15 @@ function Copy-Recursive {
|
||||
return $false
|
||||
}
|
||||
|
||||
function Is-RecurseAllowed {
|
||||
param([string]$RelativePath)
|
||||
if (-not $hasRecurseWhitelist) { return $true }
|
||||
foreach ($allow in $AllowRecursePaths) {
|
||||
if ($RelativePath.StartsWith($allow)) { return $true }
|
||||
}
|
||||
return $false
|
||||
}
|
||||
|
||||
function Process-File {
|
||||
param(
|
||||
[string]$RelativePath,
|
||||
@@ -107,17 +135,26 @@ function Copy-Recursive {
|
||||
return
|
||||
}
|
||||
|
||||
# Determine if we should recurse into this file's imports
|
||||
$shouldRecurse = $IsInitialSource -or (Is-RecurseAllowed -RelativePath $RelativePath)
|
||||
|
||||
$srcFull = Join-Path $ProjectRoot (Join-Path $McRoot $RelativePath)
|
||||
$dstFull = Join-Path $ProjectRoot (Join-Path $CoreRoot $RelativePath)
|
||||
|
||||
# Skip if already exists in core
|
||||
if (Test-Path $dstFull) {
|
||||
[void]$script:skippedExisting.Add($RelativePath)
|
||||
# STILL parse imports to find transitive deps that may not exist in core yet
|
||||
$content = [IO.File]::ReadAllText($dstFull)
|
||||
$indent = ' ' * $script:depth
|
||||
Write-Host "${indent}EXISTS: $RelativePath"
|
||||
Parse-And-Follow -Content $content
|
||||
if ($shouldRecurse) {
|
||||
# STILL parse imports to find transitive deps that may not exist in core yet
|
||||
$content = [IO.File]::ReadAllText($dstFull)
|
||||
Write-Host "${indent}EXISTS: $RelativePath"
|
||||
Parse-And-Follow -Content $content
|
||||
} else {
|
||||
Write-Host "${indent}EXISTS: $RelativePath (recursion restricted -- not in AllowRecursePaths)"
|
||||
[void]$script:skippedRecurse.Add($RelativePath)
|
||||
$script:nonRecurseImportCount++
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
@@ -134,11 +171,24 @@ function Copy-Recursive {
|
||||
|
||||
[void]$script:copied.Add($RelativePath)
|
||||
$indent = ' ' * $script:depth
|
||||
Write-Host "${indent}COPIED: $RelativePath"
|
||||
|
||||
# Parse imports and recurse
|
||||
$content = [IO.File]::ReadAllText($srcFull)
|
||||
Parse-And-Follow -Content $content
|
||||
if ($IsInitialSource) {
|
||||
$script:initialSourceCount++
|
||||
Write-Host "${indent}COPIED: $RelativePath [initial source]"
|
||||
} elseif ($shouldRecurse) {
|
||||
Write-Host "${indent}COPIED: $RelativePath"
|
||||
} else {
|
||||
Write-Host "${indent}COPIED: $RelativePath (non-recurse import -- path not in AllowRecursePaths)"
|
||||
$script:nonRecurseImportCount++
|
||||
}
|
||||
|
||||
# Parse imports and recurse (only if allowed)
|
||||
if ($shouldRecurse) {
|
||||
$content = [IO.File]::ReadAllText($srcFull)
|
||||
Parse-And-Follow -Content $content
|
||||
} else {
|
||||
[void]$script:skippedRecurse.Add($RelativePath)
|
||||
}
|
||||
}
|
||||
|
||||
function Parse-And-Follow {
|
||||
@@ -201,10 +251,16 @@ function Copy-Recursive {
|
||||
Write-Host "Dest root: $CoreRoot"
|
||||
Write-Host "Targets: $($Sources.Count) file(s)"
|
||||
if ($hasWhitelist) {
|
||||
Write-Host "Whitelist: $($AllowPaths.Count) path prefix(es)"
|
||||
Write-Host "AllowPaths: $($AllowPaths.Count) path prefix(es) [copy destination whitelist]"
|
||||
foreach ($a in $AllowPaths) { Write-Host " >> $a" }
|
||||
} else {
|
||||
Write-Host "Whitelist: NONE (all project files allowed)"
|
||||
Write-Host "AllowPaths: NONE (all project files allowed)"
|
||||
}
|
||||
if ($hasRecurseWhitelist) {
|
||||
Write-Host "AllowRecursePaths: $($AllowRecursePaths.Count) prefix(es) [recursion source whitelist]"
|
||||
foreach ($a in $AllowRecursePaths) { Write-Host " ~> $a" }
|
||||
} else {
|
||||
Write-Host "AllowRecursePaths: NONE (all discovered files recursed)"
|
||||
}
|
||||
Write-Host "============================================"
|
||||
Write-Host ""
|
||||
@@ -215,10 +271,13 @@ function Copy-Recursive {
|
||||
}
|
||||
|
||||
# ===== SUMMARY =====
|
||||
$recursivelyDiscovered = $script:copied.Count - $script:initialSourceCount
|
||||
Write-Host ""
|
||||
Write-Host "============================================"
|
||||
Write-Host " RECURSIVE COPY SUMMARY"
|
||||
Write-Host "============================================"
|
||||
Write-Host "Copied: $($script:copied.Count) files ($($script:initialSourceCount) initial sources, $($recursivelyDiscovered) recursively discovered, $($script:nonRecurseImportCount) non-recurse imports)"
|
||||
Write-Host ""
|
||||
Write-Host "Files COPIED: $($script:copied.Count)"
|
||||
foreach ($f in $script:copied) {
|
||||
Write-Host " + $f"
|
||||
@@ -233,6 +292,13 @@ function Copy-Recursive {
|
||||
foreach ($f in $script:skippedNotAllowed) {
|
||||
Write-Host " x $f"
|
||||
}
|
||||
if ($hasRecurseWhitelist) {
|
||||
Write-Host ""
|
||||
Write-Host "Recursion restricted: $($script:skippedRecurse.Count) (copied but imports NOT followed)"
|
||||
foreach ($f in $script:skippedRecurse) {
|
||||
Write-Host " ~ $f -- path not in AllowRecursePaths"
|
||||
}
|
||||
}
|
||||
Write-Host ""
|
||||
Write-Host "Not found in mc1710: $($script:notFound.Count)"
|
||||
foreach ($f in $script:notFound) {
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,799 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
pa_analyzer.py — PA Usage Analyzer for MC1710 Codebase
|
||||
|
||||
Analyzes which MC types in mc1710/ use which Platform-Abstraction (PA) members.
|
||||
|
||||
Usage:
|
||||
python tools/migration/pa_analyzer.py # dry-run
|
||||
python tools/migration/pa_analyzer.py --execute # run analysis
|
||||
python tools/migration/pa_analyzer.py --execute --pa INbt --pa IPlayer # filter
|
||||
python tools/migration/pa_analyzer.py --execute --json # machine output
|
||||
python tools/migration/pa_analyzer.py --execute --verbose # detailed
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
DEFAULT_CLASSIFICATIONS_PATH = "tools/migration/classification_results.json"
|
||||
DEFAULT_INDEX_PATH = "tools/migration/mc_usage_index.json"
|
||||
DEFAULT_OUTPUT_PATH = "tools/migration/pa_usage_analysis.json"
|
||||
|
||||
NON_SUPPRESSED_CLASSIFICATIONS = {"EXISTING", "INTERFACE", "SERVICE", "AUTO"}
|
||||
|
||||
CONFIDENCE_HIGH = 90
|
||||
CONFIDENCE_MEDIUM = 50
|
||||
|
||||
|
||||
def detect_project_root(start=None):
|
||||
cur = os.path.abspath(start or os.path.dirname(os.path.abspath(__file__)))
|
||||
for _ in range(20):
|
||||
if os.path.isfile(os.path.join(cur, "settings.gradle")):
|
||||
return cur
|
||||
parent = os.path.dirname(cur)
|
||||
if parent == cur:
|
||||
break
|
||||
cur = parent
|
||||
return os.path.abspath(start or os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
|
||||
def load_json_file(project_root, rel_path, description):
|
||||
full_path = os.path.join(project_root, rel_path)
|
||||
if not os.path.isfile(full_path):
|
||||
raise FileNotFoundError(f"{description} not found: {full_path}")
|
||||
with open(full_path, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def extract_non_suppressed_pas(
|
||||
classifications_data, classification_filter=None, pa_filter=None
|
||||
):
|
||||
types_dict = classifications_data.get("types", {})
|
||||
pa_map = {}
|
||||
|
||||
for fqn, info in types_dict.items():
|
||||
classification = info.get("classification", "")
|
||||
pa_name = info.get("pa_name") or info.get("service_name")
|
||||
|
||||
if not pa_name:
|
||||
continue
|
||||
if classification not in NON_SUPPRESSED_CLASSIFICATIONS:
|
||||
continue
|
||||
if classification_filter and classification not in classification_filter:
|
||||
continue
|
||||
if pa_filter and pa_name not in pa_filter:
|
||||
continue
|
||||
|
||||
if pa_name not in pa_map:
|
||||
pa_map[pa_name] = {"mc_types": [], "classifications": set()}
|
||||
|
||||
pa_map[pa_name]["mc_types"].append(
|
||||
{
|
||||
"fqn": fqn,
|
||||
"shortName": info.get("shortName", ""),
|
||||
"classification": classification,
|
||||
"import_count": info.get("import_count", 0),
|
||||
}
|
||||
)
|
||||
pa_map[pa_name]["classifications"].add(classification)
|
||||
|
||||
for pa_name in pa_map:
|
||||
pa_map[pa_name]["classifications"] = sorted(pa_map[pa_name]["classifications"])
|
||||
pa_map[pa_name]["mc_types"].sort(key=lambda x: -x["import_count"])
|
||||
|
||||
return pa_map
|
||||
|
||||
|
||||
def _ensure_member(member_dict, name):
|
||||
if name not in member_dict:
|
||||
member_dict[name] = {"count": 0, "files": set()}
|
||||
return member_dict[name]
|
||||
|
||||
|
||||
def _accumulate_members(member_dict, data_source, all_files, total_usage_list):
|
||||
for name, info in data_source.items():
|
||||
count = info.get("count", 0)
|
||||
files = info.get("files", [])
|
||||
entry = _ensure_member(member_dict, name)
|
||||
entry["count"] += count
|
||||
entry["files"].update(files)
|
||||
all_files.update(files)
|
||||
total_usage_list[0] += count
|
||||
|
||||
|
||||
def _accumulate_usage(
|
||||
target_count_files, usage_data, usage_key, all_files, total_usage_list
|
||||
):
|
||||
kind_entries = usage_data.get(usage_key, [])
|
||||
if isinstance(kind_entries, list):
|
||||
# Index format: array of {file, line} objects (constructors, casts, instanceof)
|
||||
for entry in kind_entries:
|
||||
f = entry.get("file", "") if isinstance(entry, dict) else ""
|
||||
if f:
|
||||
target_count_files["files"].add(f)
|
||||
all_files.add(f)
|
||||
target_count_files["count"] += len(kind_entries)
|
||||
total_usage_list[0] += len(kind_entries)
|
||||
elif isinstance(kind_entries, dict):
|
||||
# Fallback: {count, files} dict format (methodCalls, staticCalls, fieldAccesses)
|
||||
kind_count = kind_entries.get("count", 0)
|
||||
kind_files = kind_entries.get("files", [])
|
||||
target_count_files["count"] += kind_count
|
||||
target_count_files["files"].update(kind_files)
|
||||
all_files.update(kind_files)
|
||||
total_usage_list[0] += kind_count
|
||||
|
||||
|
||||
def analyze_pa_usage(pa_map, index_data):
|
||||
index_types = index_data.get("types", {})
|
||||
results = {}
|
||||
|
||||
for pa_name, pa_info in pa_map.items():
|
||||
methods = {}
|
||||
static_methods = {}
|
||||
fields = {}
|
||||
all_files = set()
|
||||
total_usage = [0]
|
||||
total_import_count = 0
|
||||
constructors = {"count": 0, "files": set()}
|
||||
casts = {"count": 0, "files": set()}
|
||||
instanceof_usage = {"count": 0, "files": set()}
|
||||
|
||||
for mc_type in pa_info["mc_types"]:
|
||||
fqn = mc_type["fqn"]
|
||||
total_import_count += mc_type["import_count"]
|
||||
type_data = index_types.get(fqn, {})
|
||||
|
||||
_accumulate_members(
|
||||
methods, type_data.get("methodCalls", {}), all_files, total_usage
|
||||
)
|
||||
_accumulate_members(
|
||||
static_methods, type_data.get("staticCalls", {}), all_files, total_usage
|
||||
)
|
||||
_accumulate_members(
|
||||
fields, type_data.get("fieldAccesses", {}), all_files, total_usage
|
||||
)
|
||||
|
||||
usage = type_data.get("usage", {})
|
||||
_accumulate_usage(constructors, usage, "new", all_files, total_usage)
|
||||
_accumulate_usage(casts, usage, "cast", all_files, total_usage)
|
||||
_accumulate_usage(
|
||||
instanceof_usage, usage, "instanceof", all_files, total_usage
|
||||
)
|
||||
|
||||
unique_members = len(methods) + len(static_methods) + len(fields)
|
||||
confidence = _compute_confidence(
|
||||
total_usage=total_usage[0],
|
||||
unique_members=unique_members,
|
||||
file_count=len(all_files),
|
||||
total_import_count=total_import_count,
|
||||
)
|
||||
|
||||
results[pa_name] = {
|
||||
"mc_types": pa_info["mc_types"],
|
||||
"classifications": pa_info["classifications"],
|
||||
"members": {
|
||||
"methods": _finalize_member_dict(methods),
|
||||
"static_methods": _finalize_member_dict(static_methods),
|
||||
"fields": _finalize_member_dict(fields),
|
||||
},
|
||||
"source_files": sorted(all_files),
|
||||
"usage_count": total_usage[0],
|
||||
"total_import_count": total_import_count,
|
||||
"unique_members": unique_members,
|
||||
"file_count": len(all_files),
|
||||
"constructors": _finalize_usage_entry(constructors),
|
||||
"casts": _finalize_usage_entry(casts),
|
||||
"instanceof": _finalize_usage_entry(instanceof_usage),
|
||||
"confidence": round(confidence, 1),
|
||||
}
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def _finalize_member_dict(member_dict):
|
||||
result = {}
|
||||
for name, info in sorted(member_dict.items(), key=lambda x: -x[1]["count"]):
|
||||
result[name] = {
|
||||
"count": info["count"],
|
||||
"file_count": len(info["files"]),
|
||||
"files": sorted(info["files"]),
|
||||
}
|
||||
return result
|
||||
|
||||
|
||||
def _finalize_usage_entry(entry):
|
||||
return {
|
||||
"count": entry["count"],
|
||||
"file_count": len(entry["files"]),
|
||||
"files": sorted(entry["files"]),
|
||||
}
|
||||
|
||||
|
||||
def _compute_confidence(total_usage, unique_members, file_count, total_import_count):
|
||||
"""Weighted score from usage density, member coverage, and file spread."""
|
||||
if total_import_count == 0:
|
||||
return 0.0
|
||||
|
||||
usage_density = min(total_usage / max(total_import_count, 1), 5.0) / 5.0
|
||||
member_score = min(unique_members / 20.0, 1.0)
|
||||
file_score = min(file_count / 50.0, 1.0)
|
||||
|
||||
return min(usage_density * 40 + member_score * 35 + file_score * 25, 100.0)
|
||||
|
||||
|
||||
def build_output(results, args):
|
||||
all_files = set()
|
||||
for r in results.values():
|
||||
all_files.update(r["source_files"])
|
||||
|
||||
metadata = {
|
||||
"analysis_date": datetime.now(timezone.utc).isoformat(),
|
||||
"tool_version": "1.0.0",
|
||||
"classifications_source": args.classifications,
|
||||
"index_source": args.index,
|
||||
"total_PAs_scanned": len(results),
|
||||
"total_mc_types_covered": sum(len(r["mc_types"]) for r in results.values()),
|
||||
"total_source_files_touched": len(all_files),
|
||||
"confidence_thresholds": {
|
||||
"high": f">={CONFIDENCE_HIGH}%",
|
||||
"medium": f">={CONFIDENCE_MEDIUM}%",
|
||||
"low": f"<{CONFIDENCE_MEDIUM}%",
|
||||
},
|
||||
"filters_applied": {
|
||||
"classifications": args.classification or "all non-suppressed",
|
||||
"pa_names": args.pa or "all",
|
||||
},
|
||||
}
|
||||
|
||||
sorted_results = dict(sorted(results.items()))
|
||||
|
||||
high_conf = []
|
||||
medium_conf = []
|
||||
low_conf = []
|
||||
by_classification = {}
|
||||
unused_pas = []
|
||||
|
||||
for pa_name, data in sorted_results.items():
|
||||
conf = data["confidence"]
|
||||
if conf >= CONFIDENCE_HIGH:
|
||||
high_conf.append(pa_name)
|
||||
elif conf >= CONFIDENCE_MEDIUM:
|
||||
medium_conf.append(pa_name)
|
||||
else:
|
||||
low_conf.append(pa_name)
|
||||
|
||||
for cls in data["classifications"]:
|
||||
if cls not in by_classification:
|
||||
by_classification[cls] = []
|
||||
by_classification[cls].append(pa_name)
|
||||
|
||||
if data["usage_count"] == 0:
|
||||
unused_pas.append(pa_name)
|
||||
|
||||
top_by_usage = [
|
||||
{
|
||||
"pa_name": name,
|
||||
"usage_count": data["usage_count"],
|
||||
"file_count": data["file_count"],
|
||||
}
|
||||
for name, data in sorted(results.items(), key=lambda x: -x[1]["usage_count"])[
|
||||
:20
|
||||
]
|
||||
]
|
||||
|
||||
summary = {
|
||||
"by_confidence": {"high": high_conf, "medium": medium_conf, "low": low_conf},
|
||||
"by_classification": by_classification,
|
||||
"top_by_usage": top_by_usage,
|
||||
"unused_pas": unused_pas,
|
||||
}
|
||||
|
||||
return {"metadata": metadata, "summary": summary, "pa_usage": sorted_results}
|
||||
|
||||
|
||||
def format_human_readable(output_data, verbose=False):
|
||||
lines = []
|
||||
meta = output_data["metadata"]
|
||||
summary = output_data["summary"]
|
||||
pa_usage = output_data["pa_usage"]
|
||||
|
||||
lines.append("")
|
||||
lines.append("=" * 72)
|
||||
lines.append("PA USAGE ANALYSIS — MC1710 Member Usage by Platform-Abstraction")
|
||||
lines.append("=" * 72)
|
||||
lines.append(f" Date: {meta['analysis_date']}")
|
||||
lines.append(f" PAs scanned: {meta['total_PAs_scanned']}")
|
||||
lines.append(f" MC types covered: {meta['total_mc_types_covered']}")
|
||||
lines.append(f" Source files touched: {meta['total_source_files_touched']}")
|
||||
lines.append("")
|
||||
|
||||
high = summary["by_confidence"]["high"]
|
||||
med = summary["by_confidence"]["medium"]
|
||||
low = summary["by_confidence"]["low"]
|
||||
lines.append(" CONFIDENCE DISTRIBUTION:")
|
||||
lines.append(f" High (>={CONFIDENCE_HIGH}%): {len(high)} PAs")
|
||||
lines.append(f" Medium (>={CONFIDENCE_MEDIUM}%): {len(med)} PAs")
|
||||
lines.append(f" Low (<{CONFIDENCE_MEDIUM}%): {len(low)} PAs")
|
||||
lines.append(f" Unused (0 usage): {len(summary['unused_pas'])} PAs")
|
||||
lines.append("")
|
||||
|
||||
lines.append(" TOP 20 PAs BY USAGE:")
|
||||
for entry in summary["top_by_usage"]:
|
||||
lines.append(
|
||||
f" {entry['pa_name']:35s} usage={entry['usage_count']:6d} files={entry['file_count']:4d}"
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
if verbose:
|
||||
lines.append("-" * 72)
|
||||
lines.append("DETAILED PA BREAKDOWN:")
|
||||
lines.append("-" * 72)
|
||||
|
||||
for pa_name, data in pa_usage.items():
|
||||
lines.append("")
|
||||
lines.append(f" PA: {pa_name}")
|
||||
lines.append(f" Classifications: {', '.join(data['classifications'])}")
|
||||
lines.append(
|
||||
f" MC Types: {', '.join(t['shortName'] for t in data['mc_types'])}"
|
||||
)
|
||||
lines.append(f" Total Usage: {data['usage_count']}")
|
||||
lines.append(f" Unique Members: {data['unique_members']}")
|
||||
lines.append(f" Files: {data['file_count']}")
|
||||
lines.append(f" Confidence: {data['confidence']}%")
|
||||
|
||||
methods = data["members"]["methods"]
|
||||
if methods:
|
||||
lines.append(f" Instance Methods ({len(methods)}):")
|
||||
for name, info in list(methods.items())[:10]:
|
||||
lines.append(
|
||||
f" {name:40s} {info['count']:5d} calls {info['file_count']:3d} files"
|
||||
)
|
||||
if len(methods) > 10:
|
||||
lines.append(f" ... and {len(methods) - 10} more")
|
||||
|
||||
statics = data["members"]["static_methods"]
|
||||
if statics:
|
||||
lines.append(f" Static Methods ({len(statics)}):")
|
||||
for name, info in list(statics.items())[:5]:
|
||||
lines.append(
|
||||
f" {name:40s} {info['count']:5d} calls {info['file_count']:3d} files"
|
||||
)
|
||||
if len(statics) > 5:
|
||||
lines.append(f" ... and {len(statics) - 5} more")
|
||||
|
||||
fields_data = data["members"]["fields"]
|
||||
if fields_data:
|
||||
lines.append(f" Fields ({len(fields_data)}):")
|
||||
for name, info in list(fields_data.items())[:10]:
|
||||
lines.append(
|
||||
f" {name:40s} {info['count']:5d} accesses {info['file_count']:3d} files"
|
||||
)
|
||||
if len(fields_data) > 10:
|
||||
lines.append(f" ... and {len(fields_data) - 10} more")
|
||||
|
||||
if data["constructors"]["count"] > 0:
|
||||
lines.append(
|
||||
f" Constructors: {data['constructors']['count']} in {data['constructors']['file_count']} files"
|
||||
)
|
||||
if data["casts"]["count"] > 0:
|
||||
lines.append(
|
||||
f" Casts: {data['casts']['count']} in {data['casts']['file_count']} files"
|
||||
)
|
||||
if data["instanceof"]["count"] > 0:
|
||||
lines.append(
|
||||
f" instanceof: {data['instanceof']['count']} in {data['instanceof']['file_count']} files"
|
||||
)
|
||||
|
||||
if summary["unused_pas"]:
|
||||
lines.append("")
|
||||
lines.append(" UNUSED PAs (0 member usage detected in index):")
|
||||
for i in range(0, len(summary["unused_pas"]), 6):
|
||||
chunk = summary["unused_pas"][i : i + 6]
|
||||
lines.append(f" {', '.join(chunk)}")
|
||||
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _print_dry_run(pa_map, args):
|
||||
print("")
|
||||
print("=" * 72)
|
||||
print("PA USAGE ANALYZER — DRY RUN (add --execute to run)")
|
||||
print("=" * 72)
|
||||
print(f" Classifications source: {args.classifications}")
|
||||
print(f" Index source: {args.index}")
|
||||
print(f" Output target: {args.output}")
|
||||
print(f" PA filter: {args.pa or 'all'}")
|
||||
print(f" Classification filter: {args.classification or 'all non-suppressed'}")
|
||||
print(f"")
|
||||
print(f" Non-suppressed PAs found: {len(pa_map)}")
|
||||
total_mc = sum(len(v["mc_types"]) for v in pa_map.values())
|
||||
print(f" Total MC types to scan: {total_mc}")
|
||||
print(f"")
|
||||
|
||||
by_class = {}
|
||||
for pa_name, info in sorted(pa_map.items()):
|
||||
for cls in info["classifications"]:
|
||||
if cls not in by_class:
|
||||
by_class[cls] = []
|
||||
by_class[cls].append(pa_name)
|
||||
|
||||
for cls in sorted(by_class):
|
||||
pas = by_class[cls]
|
||||
print(f" {cls} ({len(pas)} PAs):")
|
||||
for pa_name in pas[:10]:
|
||||
mc_types = pa_map[pa_name]["mc_types"]
|
||||
mc_str = ", ".join(t["shortName"] for t in mc_types[:3])
|
||||
if len(mc_types) > 3:
|
||||
mc_str += f" +{len(mc_types) - 3} more"
|
||||
total_imports = sum(t["import_count"] for t in mc_types)
|
||||
print(f" {pa_name:35s} mc=[{mc_str}] imports={total_imports}")
|
||||
if len(pas) > 10:
|
||||
print(f" ... and {len(pas) - 10} more")
|
||||
print("")
|
||||
|
||||
print(" Run with --execute to perform the full analysis.")
|
||||
print("")
|
||||
|
||||
|
||||
def export_all_methods(
|
||||
index_data,
|
||||
output_path,
|
||||
classifications_data=None,
|
||||
classification_filter=None,
|
||||
as_json=False,
|
||||
):
|
||||
index_types = index_data.get("types", {})
|
||||
|
||||
allowed_fqns = None
|
||||
fqn_to_classification = {}
|
||||
|
||||
if classifications_data:
|
||||
types_dict = classifications_data.get("types", {})
|
||||
allowed_fqns = set()
|
||||
for fqn, info in types_dict.items():
|
||||
cls = info.get("classification", "")
|
||||
if classification_filter and cls not in classification_filter:
|
||||
continue
|
||||
allowed_fqns.add(fqn)
|
||||
fqn_to_classification[fqn] = cls
|
||||
|
||||
def _sort_members(members):
|
||||
return sorted(members, key=lambda x: (-x["count"], x["name"]))
|
||||
|
||||
types_output = []
|
||||
total_method_usage = 0
|
||||
total_field_usage = 0
|
||||
total_static_usage = 0
|
||||
total_methods = 0
|
||||
total_fields = 0
|
||||
total_statics = 0
|
||||
per_classification_stats = {}
|
||||
|
||||
for fqn, type_data in sorted(index_types.items()):
|
||||
if allowed_fqns is not None and fqn not in allowed_fqns:
|
||||
continue
|
||||
|
||||
short_name = type_data.get("shortName", "")
|
||||
fqn_cls = fqn_to_classification.get(fqn, "UNKNOWN")
|
||||
|
||||
methods = []
|
||||
for name, info in type_data.get("methodCalls", {}).items():
|
||||
count = info.get("count", 0)
|
||||
files = info.get("files", [])
|
||||
methods.append(
|
||||
{"name": name, "count": count, "file_count": len(files), "files": files}
|
||||
)
|
||||
total_method_usage += count
|
||||
|
||||
fields = []
|
||||
for name, info in type_data.get("fieldAccesses", {}).items():
|
||||
count = info.get("count", 0)
|
||||
files = info.get("files", [])
|
||||
fields.append(
|
||||
{"name": name, "count": count, "file_count": len(files), "files": files}
|
||||
)
|
||||
total_field_usage += count
|
||||
|
||||
statics = []
|
||||
for name, info in type_data.get("staticCalls", {}).items():
|
||||
count = info.get("count", 0)
|
||||
files = info.get("files", [])
|
||||
statics.append(
|
||||
{"name": name, "count": count, "file_count": len(files), "files": files}
|
||||
)
|
||||
total_static_usage += count
|
||||
|
||||
methods = _sort_members(methods)
|
||||
fields = _sort_members(fields)
|
||||
statics = _sort_members(statics)
|
||||
|
||||
total_methods += len(methods)
|
||||
total_fields += len(fields)
|
||||
total_statics += len(statics)
|
||||
type_usage = (
|
||||
sum(m["count"] for m in methods)
|
||||
+ sum(f["count"] for f in fields)
|
||||
+ sum(s["count"] for s in statics)
|
||||
)
|
||||
|
||||
type_entry = {
|
||||
"fqn": fqn,
|
||||
"shortName": short_name,
|
||||
"classification": fqn_cls,
|
||||
"import_count": type_data.get("importCount", 0),
|
||||
"total_usage": type_usage,
|
||||
"methods": methods,
|
||||
"fields": fields,
|
||||
"static_methods": statics,
|
||||
}
|
||||
types_output.append(type_entry)
|
||||
|
||||
if fqn_cls not in per_classification_stats:
|
||||
per_classification_stats[fqn_cls] = {
|
||||
"types": 0,
|
||||
"methods": 0,
|
||||
"fields": 0,
|
||||
"statics": 0,
|
||||
"usage": 0,
|
||||
}
|
||||
per_classification_stats[fqn_cls]["types"] += 1
|
||||
per_classification_stats[fqn_cls]["methods"] += len(methods)
|
||||
per_classification_stats[fqn_cls]["fields"] += len(fields)
|
||||
per_classification_stats[fqn_cls]["statics"] += len(statics)
|
||||
per_classification_stats[fqn_cls]["usage"] += type_usage
|
||||
|
||||
filter_label = "ALL (unfiltered)"
|
||||
if classification_filter:
|
||||
filter_label = ", ".join(sorted(classification_filter))
|
||||
|
||||
output = {
|
||||
"metadata": {
|
||||
"export_date": datetime.now(timezone.utc).isoformat(),
|
||||
"classification_filter": filter_label,
|
||||
"total_mc_types": len(types_output),
|
||||
"total_methods": total_methods,
|
||||
"total_fields": total_fields,
|
||||
"total_statics": total_statics,
|
||||
"total_unique_members": total_methods + total_fields + total_statics,
|
||||
"total_method_usage": total_method_usage,
|
||||
"total_field_usage": total_field_usage,
|
||||
"total_static_usage": total_static_usage,
|
||||
"total_usage": total_method_usage + total_field_usage + total_static_usage,
|
||||
"per_classification": {
|
||||
cls: stats for cls, stats in sorted(per_classification_stats.items())
|
||||
},
|
||||
},
|
||||
"types": types_output,
|
||||
}
|
||||
|
||||
os.makedirs(os.path.dirname(os.path.abspath(output_path)), exist_ok=True)
|
||||
with open(output_path, "w", encoding="utf-8", newline="\n") as f:
|
||||
json.dump(output, f, indent=2, ensure_ascii=False)
|
||||
|
||||
meta = output["metadata"]
|
||||
if as_json:
|
||||
print(json.dumps(meta, indent=2))
|
||||
else:
|
||||
print("")
|
||||
print("=" * 72)
|
||||
print(f"ALL METHODS EXPORT — Classifications: {filter_label}")
|
||||
print("=" * 72)
|
||||
print(f" MC types scanned: {meta['total_mc_types']}")
|
||||
print(
|
||||
f" Unique methods: {meta['total_methods']:,} ({meta['total_method_usage']:,} total calls)"
|
||||
)
|
||||
print(
|
||||
f" Unique fields: {meta['total_fields']:,} ({meta['total_field_usage']:,} total accesses)"
|
||||
)
|
||||
print(
|
||||
f" Unique statics: {meta['total_statics']:,} ({meta['total_static_usage']:,} total calls)"
|
||||
)
|
||||
print(f" Total unique members:{meta['total_unique_members']:,}")
|
||||
print(f" Total usage: {meta['total_usage']:,}")
|
||||
|
||||
if per_classification_stats:
|
||||
print("")
|
||||
print(" PER-CLASSIFICATION BREAKDOWN:")
|
||||
for cls in sorted(per_classification_stats):
|
||||
s = per_classification_stats[cls]
|
||||
print(
|
||||
f" {cls:12s} {s['types']:3d} types {s['methods']:4d} methods {s['fields']:4d} fields {s['statics']:4d} statics {s['usage']:6d} usage"
|
||||
)
|
||||
|
||||
top_types = sorted(types_output, key=lambda t: -t["total_usage"])[:20]
|
||||
print("")
|
||||
print(" TOP 20 TYPES BY USAGE:")
|
||||
for t in top_types:
|
||||
print(
|
||||
f" {t['shortName']:30s} [{t['classification']:9s}] {t['total_usage']:6d} usage {len(t['methods']):3d}m {len(t['fields']):3d}f {len(t['static_methods']):3d}s"
|
||||
)
|
||||
|
||||
print("")
|
||||
print(f" Output written to: {output_path}")
|
||||
print("")
|
||||
|
||||
return output
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="PA Usage Analyzer — Maps PA member usage across mc1710/ source files.",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
python tools/migration/pa_analyzer.py # dry-run
|
||||
python tools/migration/pa_analyzer.py --execute # run analysis
|
||||
python tools/migration/pa_analyzer.py --execute --pa INbt --pa IPlayer
|
||||
python tools/migration/pa_analyzer.py --execute --classification EXISTING INTERFACE
|
||||
python tools/migration/pa_analyzer.py --execute --classification SERVICE
|
||||
python tools/migration/pa_analyzer.py --execute --output /tmp/pa_usage.json
|
||||
python tools/migration/pa_analyzer.py --execute --verbose
|
||||
python tools/migration/pa_analyzer.py --execute --json
|
||||
python tools/migration/pa_analyzer.py --all-methods # export all MC members
|
||||
python tools/migration/pa_analyzer.py --all-methods --json # JSON metadata only
|
||||
""",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--execute",
|
||||
action="store_true",
|
||||
help="Run analysis and write output (default: dry-run)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output",
|
||||
default=None,
|
||||
help=f"Output JSON path (default: {DEFAULT_OUTPUT_PATH})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--classifications",
|
||||
default=DEFAULT_CLASSIFICATIONS_PATH,
|
||||
help=f"Classification results JSON (default: {DEFAULT_CLASSIFICATIONS_PATH})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--index",
|
||||
default=DEFAULT_INDEX_PATH,
|
||||
help=f"MC usage index JSON (default: {DEFAULT_INDEX_PATH})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--pa",
|
||||
action="append",
|
||||
default=None,
|
||||
help="Filter to specific PA name(s). Repeatable.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--classification",
|
||||
nargs="*",
|
||||
default=None,
|
||||
help="Filter by classification(s): EXISTING, INTERFACE, SERVICE, AUTO",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--json", action="store_true", help="Machine-readable JSON summary to stdout"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose", action="store_true", help="Show detailed per-PA breakdown"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--all-methods",
|
||||
action="store_true",
|
||||
help="Export MC method/field/static usage from the index, filtered by classification",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--all-methods-classification",
|
||||
nargs="*",
|
||||
default=None,
|
||||
help="Classifications for --all-methods: EXISTING INTERFACE SERVICE AUTO TRANSFORM (default: all). Use ALL explicitly to skip loading classifications file.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--project-root",
|
||||
default=None,
|
||||
help="Project root override (auto-detected by default)",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
project_root = (
|
||||
os.path.abspath(args.project_root)
|
||||
if args.project_root
|
||||
else detect_project_root()
|
||||
)
|
||||
if args.output is None:
|
||||
args.output = DEFAULT_OUTPUT_PATH
|
||||
|
||||
try:
|
||||
if args.all_methods:
|
||||
index_data = load_json_file(project_root, args.index, "Index")
|
||||
all_methods_output = args.output.replace(".json", "_all_methods.json")
|
||||
if args.output == DEFAULT_OUTPUT_PATH:
|
||||
all_methods_output = "tools/migration/all_methods_export.json"
|
||||
full_output_path = os.path.join(project_root, all_methods_output)
|
||||
|
||||
cls_data = None
|
||||
cls_filter = None
|
||||
if args.all_methods_classification and "ALL" in [
|
||||
c.upper() for c in args.all_methods_classification
|
||||
]:
|
||||
pass
|
||||
else:
|
||||
cls_data = load_json_file(
|
||||
project_root, args.classifications, "Classifications"
|
||||
)
|
||||
if args.all_methods_classification:
|
||||
cls_filter = set(c.upper() for c in args.all_methods_classification)
|
||||
|
||||
export_all_methods(
|
||||
index_data,
|
||||
full_output_path,
|
||||
classifications_data=cls_data,
|
||||
classification_filter=cls_filter,
|
||||
as_json=args.json,
|
||||
)
|
||||
return
|
||||
|
||||
classifications_data = load_json_file(
|
||||
project_root, args.classifications, "Classifications"
|
||||
)
|
||||
classification_filter = (
|
||||
set(args.classification) if args.classification else None
|
||||
)
|
||||
pa_filter = set(args.pa) if args.pa else None
|
||||
|
||||
pa_map = extract_non_suppressed_pas(
|
||||
classifications_data,
|
||||
classification_filter=classification_filter,
|
||||
pa_filter=pa_filter,
|
||||
)
|
||||
|
||||
if not args.execute:
|
||||
_print_dry_run(pa_map, args)
|
||||
return
|
||||
|
||||
index_data = load_json_file(project_root, args.index, "Index")
|
||||
results = analyze_pa_usage(pa_map, index_data)
|
||||
|
||||
output_data = build_output(results, args)
|
||||
|
||||
output_path = os.path.join(project_root, args.output)
|
||||
os.makedirs(os.path.dirname(output_path), exist_ok=True)
|
||||
with open(output_path, "w", encoding="utf-8", newline="\n") as f:
|
||||
json.dump(output_data, f, indent=2, ensure_ascii=False)
|
||||
|
||||
if args.json:
|
||||
conf_data = output_data["summary"]["by_confidence"]
|
||||
summary_output = {
|
||||
"status": "success",
|
||||
"output_file": args.output,
|
||||
"total_pas": len(results),
|
||||
"total_mc_types": sum(len(r["mc_types"]) for r in results.values()),
|
||||
"total_files": output_data["metadata"]["total_source_files_touched"],
|
||||
"confidence_distribution": {
|
||||
"high": len(conf_data["high"]),
|
||||
"medium": len(conf_data["medium"]),
|
||||
"low": len(conf_data["low"]),
|
||||
},
|
||||
"unused_pas": len(output_data["summary"]["unused_pas"]),
|
||||
}
|
||||
print(json.dumps(summary_output, indent=2))
|
||||
else:
|
||||
print(format_human_readable(output_data, verbose=args.verbose))
|
||||
print(f" Output written to: {args.output}")
|
||||
print("")
|
||||
|
||||
except FileNotFoundError as e:
|
||||
print(f"ERROR: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
except json.JSONDecodeError as e:
|
||||
print(f"ERROR: Invalid JSON — {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
print(f"ERROR: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,149 @@
|
||||
# test_allow_recurse_paths.ps1
|
||||
# ============================
|
||||
# Test script for the -AllowRecursePaths parameter on Copy-Recursive.
|
||||
#
|
||||
# Runs 3 scenarios against REAL codebase files:
|
||||
# Scenario A: No AllowRecursePaths (all imports followed - current/default behavior)
|
||||
# Scenario B: AllowRecursePaths = kamkeel/npcs/controllers only
|
||||
# Scenario C: AllowRecursePaths = noppes/npcs/controllers + noppes/npcs/util
|
||||
#
|
||||
# Each scenario copies to a temporary core directory, counts results, and cleans up.
|
||||
# Execute: powershell -ExecutionPolicy Bypass -File "tools/migration/test_allow_recurse_paths.ps1"
|
||||
|
||||
. (Join-Path $PSScriptRoot 'Copy-Recursive.ps1')
|
||||
|
||||
$ProjectRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..')).Path
|
||||
$McRoot = 'mc1710/src/main/java'
|
||||
$TestCoreBase = Join-Path $ProjectRoot '_test_allow_recurse_paths'
|
||||
|
||||
# Cleanup from previous runs
|
||||
if (Test-Path $TestCoreBase) {
|
||||
Remove-Item $TestCoreBase -Recurse -Force
|
||||
}
|
||||
|
||||
# Common sources and AllowPaths for all scenarios
|
||||
# We use DialogController because it imports from many different packages:
|
||||
# - kamkeel/npcs/controllers/SyncController
|
||||
# - noppes/npcs/controllers/data/Dialog, DialogCategory, DialogOption
|
||||
# - noppes/npcs/constants/EnumOptionType
|
||||
# - noppes/npcs/util/NBTJsonUtil
|
||||
# - noppes/npcs/CustomNpcs, noppes/npcs/LogWriter, noppes/npcs/NoppesStringUtils
|
||||
$Sources = @(
|
||||
'noppes/npcs/controllers/DialogController.java'
|
||||
)
|
||||
# Wide AllowPaths so files from many directories CAN be copied (destination-allowed)
|
||||
$AllowPaths = @(
|
||||
'noppes/npcs/controllers/',
|
||||
'noppes/npcs/controllers/data/',
|
||||
'noppes/npcs/constants/',
|
||||
'noppes/npcs/util/',
|
||||
'noppes/npcs/',
|
||||
'kamkeel/npcs/controllers/'
|
||||
)
|
||||
|
||||
function Run-Scenario {
|
||||
param(
|
||||
[string]$ScenarioName,
|
||||
[string]$Description,
|
||||
[string[]]$RecursePaths = @()
|
||||
)
|
||||
Write-Host ""
|
||||
Write-Host "================================================================" -ForegroundColor Cyan
|
||||
Write-Host (" SCENARIO {0}: {1}" -f $ScenarioName, $Description) -ForegroundColor Cyan
|
||||
Write-Host "================================================================" -ForegroundColor Cyan
|
||||
|
||||
$scenarioDir = "scenario_${ScenarioName}"
|
||||
$scenarioCore = Join-Path $TestCoreBase $scenarioDir
|
||||
New-Item -ItemType Directory -Path (Join-Path $scenarioCore 'src/main/java') -Force | Out-Null
|
||||
|
||||
$result = Copy-Recursive `
|
||||
-Sources $Sources `
|
||||
-AllowPaths $AllowPaths `
|
||||
-AllowRecursePaths $RecursePaths `
|
||||
-McRoot $McRoot `
|
||||
-CoreRoot "_test_allow_recurse_paths/${scenarioDir}/src/main/java" `
|
||||
-ProjectRoot $ProjectRoot
|
||||
|
||||
Write-Host ""
|
||||
$cnt = 0
|
||||
if ($result) { $cnt = $result.Count }
|
||||
Write-Host (" >> Scenario {0} result: {1} files copied" -f $ScenarioName, $cnt) -ForegroundColor Yellow
|
||||
Write-Host ""
|
||||
return $result
|
||||
}
|
||||
|
||||
# ================================================================
|
||||
# SCENARIO A: No AllowRecursePaths (default - all imports followed)
|
||||
# ================================================================
|
||||
$resultA = Run-Scenario -ScenarioName 'A' -Description 'No AllowRecursePaths (default behavior, all imports followed)'
|
||||
|
||||
# ================================================================
|
||||
# SCENARIO B: AllowRecursePaths = kamkeel/npcs/controllers only
|
||||
# ================================================================
|
||||
$resultB = Run-Scenario -ScenarioName 'B' `
|
||||
-Description 'AllowRecursePaths = kamkeel/npcs/controllers/ only' `
|
||||
-RecursePaths @('kamkeel/npcs/controllers/')
|
||||
|
||||
# ================================================================
|
||||
# SCENARIO C: AllowRecursePaths = noppes/npcs/controllers + noppes/npcs/util
|
||||
# ================================================================
|
||||
$resultC = Run-Scenario -ScenarioName 'C' `
|
||||
-Description 'AllowRecursePaths = noppes/npcs/controllers/ + noppes/npcs/util/' `
|
||||
-RecursePaths @('noppes/npcs/controllers/', 'noppes/npcs/util/')
|
||||
|
||||
# ================================================================
|
||||
# COMPARISON TABLE
|
||||
# ================================================================
|
||||
Write-Host ""
|
||||
Write-Host "================================================================" -ForegroundColor Green
|
||||
Write-Host " COMPARISON TABLE" -ForegroundColor Green
|
||||
Write-Host "================================================================" -ForegroundColor Green
|
||||
Write-Host ""
|
||||
$countA = 0; if ($resultA) { $countA = $resultA.Count }
|
||||
$countB = 0; if ($resultB) { $countB = $resultB.Count }
|
||||
$countC = 0; if ($resultC) { $countC = $resultC.Count }
|
||||
Write-Host (" Scenario AllowRecursePaths Files Copied") -ForegroundColor White
|
||||
Write-Host (" -------- ---------------------------------- ------------") -ForegroundColor White
|
||||
Write-Host (" A (none - all recursed) {0}" -f $countA) -ForegroundColor White
|
||||
Write-Host (" B kamkeel/npcs/controllers/ {0}" -f $countB) -ForegroundColor White
|
||||
Write-Host (" C noppes/.../controllers/ + util/ {0}" -f $countC) -ForegroundColor White
|
||||
Write-Host ""
|
||||
|
||||
# Show what's in A but not B (files that were NOT copied due to restricted recursion)
|
||||
if ($resultA -and $resultB) {
|
||||
$onlyInA_notB = $resultA | Where-Object { $resultB -notcontains $_ }
|
||||
if ($onlyInA_notB -and $onlyInA_notB.Count -gt 0) {
|
||||
Write-Host ("Files in Scenario A but NOT in Scenario B ({0} files):" -f $onlyInA_notB.Count) -ForegroundColor Yellow
|
||||
foreach ($f in $onlyInA_notB) {
|
||||
Write-Host " - $f" -ForegroundColor DarkYellow
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if ($resultA -and $resultC) {
|
||||
$onlyInA_notC = $resultA | Where-Object { $resultC -notcontains $_ }
|
||||
if ($onlyInA_notC -and $onlyInA_notC.Count -gt 0) {
|
||||
Write-Host ""
|
||||
Write-Host ("Files in Scenario A but NOT in Scenario C ({0} files):" -f $onlyInA_notC.Count) -ForegroundColor Yellow
|
||||
foreach ($f in $onlyInA_notC) {
|
||||
Write-Host " - $f" -ForegroundColor DarkYellow
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host ""
|
||||
|
||||
# Backward compat check
|
||||
Write-Host "================================================================" -ForegroundColor Green
|
||||
Write-Host " BACKWARD COMPATIBILITY CHECK" -ForegroundColor Green
|
||||
Write-Host "================================================================" -ForegroundColor Green
|
||||
Write-Host ("Scenario A (no AllowRecursePaths) = default behavior: {0} files copied" -f $countA) -ForegroundColor White
|
||||
Write-Host "This matches the behavior of Copy-Recursive WITHOUT the new parameter." -ForegroundColor White
|
||||
Write-Host ""
|
||||
|
||||
# Cleanup
|
||||
Write-Host "Cleaning up test directories..." -ForegroundColor DarkGray
|
||||
if (Test-Path $TestCoreBase) {
|
||||
Remove-Item $TestCoreBase -Recurse -Force
|
||||
}
|
||||
Write-Host "Done." -ForegroundColor DarkGray
|
||||
Reference in new issue
Block a user