Retiring Prompt Assets That Refuse to Die — A Ledger for Paths Static Analysis Cannot See

A Fixed Idle Threshold Fails in Both Directions

The other thing I went back and forth on was the dormancy rule. "Retire anything untouched for 90 days" is easy to implement and feels reasonable.

Here is the same sample ledger judged both ways — a flat 90-day rule versus a window derived from each asset's schedule.

The flat threshold missed in both directions.

A year-end summarization prompt that is idle for 210 days is perfectly healthy; the 90-day rule marks it for deletion. Meanwhile a reply-drafting prompt that should fire daily has been silent for 85 days, which is plainly broken — and the flat rule waves it through because it has not reached 90. It deletes what should stay and ignores what should be caught. That is what happens when one number is asked to serve every cadence.

What I settled on is a window relative to each asset's own schedule.

PERIOD_DAYS = {"on_demand": 30, "daily": 1, "weekly": 7, "monthly": 31, "yearly": 365}
period = PERIOD_DAYS.get(entry.get("schedule", "on_demand"), 30)
window = period * 2            # one period cannot separate "idle" from "stopped"

The multiplier of two was deliberate. At exactly one period, a run that slips by a few hours reads as dormant. At three or more, a daily asset takes nearly a week to raise an alarm. If one formula has to cover daily through annual cadences, two turned out to be the comfortable landing spot. Demanding a 730-day window for an annual asset looks excessive at first glance, but an annual batch only offers one observation opportunity per year, and I have come to treat that as a property to accept rather than a flaw to engineer around.

The Full Ledger — No First-Match Shortcuts

Here is the rewritten implementation in full. The key change is that it does not narrow down to a single classification; it returns every finding that applies.

#!/usr/bin/env python3
"""Prompt asset retirement ledger. Cross-checks static references, flag defaults,
observation windows, model EOL dates and cache effectiveness, and reports every
applicable finding rather than the first match."""
from __future__ import annotations
import json, re, sys
from datetime import datetime, timezone
from pathlib import Path
 
PERIOD_DAYS = {"on_demand": 30, "daily": 1, "weekly": 7, "monthly": 31, "yearly": 365}
SOURCE_SUFFIXES = {".py", ".ts", ".tsx", ".js", ".swift", ".kt", ".gs"}
 
# Models with a published shutdown date (verify on the ai.google.dev deprecations page)
MODEL_EOL = {
    "gemini-3.1-flash-image-preview": "2026-08-17",
    "gemini-3-pro-image-preview": "2026-08-17",
    "gemini-2.5-flash": "2026-06-01",
}
SEVERITY = {"PHANTOM": 0, "MODEL_EOL": 1, "ZOMBIE_CACHE": 2, "GATED_BUT_LIVE": 3,
            "UNREACHABLE": 4, "GATED": 5, "DORMANT_CONFIRMED": 6,
            "DORMANT_WATCH": 7, "ACTIVE": 8}
 
def scan_references(root: Path) -> dict[str, list[str]]:
    """Collect prompt IDs that appear as literals in source."""
    found: dict[str, list[str]] = {}
    for path in root.rglob("*"):
        if path.suffix not in SOURCE_SUFFIXES or not path.is_file():
            continue
        text = path.read_text(encoding="utf-8", errors="replace")
        for literal in re.findall(r"[\"']([a-z0-9_.\-]+\.[a-z0-9_\-]+)[\"']", text):
            found.setdefault(literal, []).append(str(path))
    return found
 
def audit(entry: dict, refs: dict, flags: dict, now: datetime) -> list[tuple[str, str]]:
    pid, findings = entry["id"], []
    period = PERIOD_DAYS.get(entry.get("schedule", "on_demand"), 30)
    window = period * 2
    last_hit = datetime.fromisoformat(entry["last_hit"].replace("Z", "+00:00"))
    idle = (now - last_hit).days
    referenced = pid in refs
    flag = entry.get("flag")
    flag_on = flags.get(flag, False) if flag is not None else True
    live = idle <= period
 
    # Invisible to static analysis but running. The most dangerous state in the ledger
    if not referenced and live:
        findings.append(("PHANTOM",
            f"No source reference, yet ran {idle} days ago. ID likely built dynamically"))
    elif not referenced:
        findings.append(("UNREACHABLE", f"No source reference; idle {idle} days. Retirable"))
 
    if not flag_on:
        label = "GATED_BUT_LIVE" if live else "GATED"
        findings.append((label, f"Flag {flag} defaults to false; last run {idle} days ago"))
 
    # An EOL warning on an unreachable asset is noise: deletion, not migration
    eol = MODEL_EOL.get(entry.get("model", ""))
    if eol and (referenced or live):
        findings.append(("MODEL_EOL", f"{entry['model']} shuts down {eol}. Migration deadline"))
 
    ratio = entry.get("cache_hit_ratio_30d")
    if entry.get("cached_content") and ratio is not None and ratio < 0.5:
        findings.append(("ZOMBIE_CACHE",
            f"Cache effectiveness {ratio:.0%}; billing full input on every call"))
 
    if referenced and flag_on and not live:
        if idle > window:
            findings.append(("DORMANT_CONFIRMED",
                f"Idle {idle} days, past the {window}-day window. Retirement candidate"))
        else:
            findings.append(("DORMANT_WATCH",
                f"Idle {idle} days, short of the {window}-day window. Hold"))
 
    return findings or [("ACTIVE", f"Last run {idle} days ago, within a {period}-day period")]
 
def main(root: str = ".") -> int:
    base = Path(root)
    inventory = json.loads((base / "prompt_inventory.json").read_text(encoding="utf-8"))
    flags = json.loads((base / "flags.json").read_text(encoding="utf-8"))
    refs = scan_references(base / "src")
    now = datetime.now(timezone.utc)
 
    rows = []
    for entry in inventory:
        for status, reason in audit(entry, refs, flags, now):
            rows.append((entry["id"], status, reason))
    rows.sort(key=lambda r: (SEVERITY[r[1]], r[0]))
 
    width = max(len(r[0]) for r in rows)
    for pid, status, reason in rows:
        print(f"{pid:<{width}}  {status:<18}  {reason}")
 
    blocking = {p for p, s, _ in rows if SEVERITY[s] <= 3}
    print(f"\n{len(rows)} findings across {len(inventory)} assets; {len(blocking)} need action")
    return 1 if blocking else 0
 
if __name__ == "__main__":
    sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else "."))

The input side is deliberately plain JSON. Keeping it in a shape that can be generated mechanically from existing configuration is what makes the habit survive.

[
  {"id": "wallpaper.classify.v3", "model": "gemini-3.5-flash", "flag": "wallpaper_auto_tag",
   "schedule": "daily", "cached_content": "cachedContents/wp-taxonomy",
   "cache_hit_ratio_30d": 0.94, "last_hit": "2026-07-26T03:10:00Z"},
  {"id": "wallpaper.classify.v2", "model": "gemini-3.5-flash", "flag": "wallpaper_auto_tag",
   "schedule": "daily", "cached_content": "cachedContents/wp-taxonomy",
   "cache_hit_ratio_30d": 0.11, "last_hit": "2026-07-25T03:10:00Z"},
  {"id": "tax.yearend.summary.v1", "model": "gemini-3.5-flash", "flag": null,
   "schedule": "yearly", "cached_content": "cachedContents/tax-rules",
   "cache_hit_ratio_30d": null, "last_hit": "2025-12-28T22:00:00Z"}
]

last_hit and cache_hit_ratio_30d come from a daily rollup of the call logs I write on every invocation. Now that developer logs for Interactions API calls are visible in the AI Studio dashboard, reconciling my own numbers against the platform's view is far less painful than it used to be. Whether the ledger's values deserve trust depends entirely on whether you have done that reconciliation even once.

Reading the Output

Run against the sample ledger, with 2026-07-27 as the run date and last_hit values exactly as listed above. Idle counts shift as the calendar moves, so shift last_hit forward if you want to reproduce this locally.

wallpaper.classify.v2        PHANTOM        No source reference, yet ran 1 day ago
store.screenshot.caption.v1  MODEL_EOL      gemini-3.1-flash-image-preview shuts down 2026-08-17
wallpaper.classify.v2        ZOMBIE_CACHE   Cache effectiveness 11%; billing full input
wallpaper.caption.legacy     UNREACHABLE    No source reference; idle 162 days. Retirable
review.reply.draft.v2        GATED          Flag review_autoreply defaults to false; 85 days
store.metadata.translate.v4  ACTIVE         Last run 5 days ago, within a 7-day period
tax.yearend.summary.v1       ACTIVE         Last run 210 days ago, within a 365-day period
wallpaper.classify.v3        ACTIVE         Last run 0 days ago, within a 1-day period
 
8 findings across 7 assets; 2 need action

The payoff from abandoning first-match shows up in the two rows for wallpaper.classify.v2. That asset is a phantom path and an expensive one, running at 11 percent cache effectiveness. Under the first-match classifier the cache finding never appeared at all, hidden behind UNREACHABLE. A design that narrows to one label structurally conceals every problem after the first.

There was also a finding I removed during the rewrite. Initially wallpaper.caption.legacy produced both MODEL_EOL and UNREACHABLE — it referenced a retired model and was unreachable. Being told about a migration deadline is useless when the answer is deletion, not migration.

# An EOL warning on an unreachable asset is noise: deletion, not migration
if eol and (referenced or live):

Multi-label reporting produces more findings, and every extra finding creates work to suppress the ones you do not want. "Just emit everything" is not a strategy either — something I only really felt once I was staring at the output.

The Cache Dies Before the Prompt Does

This is where my understanding shifted most.

Context caches have a TTL. The prompt asset itself can be perfectly alive, referenced from code, running on schedule — while the cache handle expires underneath it and every subsequent call pays for the full input.

Nothing about this registers as a failure. The response comes back, no error is raised, output quality is unchanged. The only thing that changes is the invoice. In indie development the revenue side gets checked daily — I open the AdMob dashboard without thinking about it — while the cost side waits for a monthly statement. That asymmetry is what lets this kind of decay survive so long. I started building this as a tool for finding things to delete, and the first thing it actually earned its keep on was the operating quality of assets I must not delete.

I put the threshold at 0.5 because below that, holding a cache stops being worth the bookkeeping. The precise break-even moves with input size and call frequency, but once effectiveness drops under half, it is faster to assume the TTL configuration or the recreation step is broken and go look, rather than to model it.

The order I work in now:

1. PHANTOM — identify the path first. Deletion decisions come later
2. ZOMBIE_CACHE / GATED_BUT_LIVE — running with a cost or intent problem. Fix before anything else
3. MODEL_EOL — work backwards from the shutdown date into a migration plan
4. UNREACHABLE / DORMANT_CONFIRMED — only here does retirement enter the conversation

Do not start from retirement. That was the clearest lesson from actually running the thing.

Putting It Into Rotation

A ledger is at its most accurate on the day you build it, and degrades from there. Two habits keep mine from decaying.

The first is running it in CI. It returns exit code 1 whenever a finding at severity 3 or below remains, so failing the job is straightforward. But I chose that boundary carefully. Failing CI on DORMANT_CONFIRMED would turn the build red every time someone comes back from a holiday, and before long nobody reads it. Restricting failures to the "running, with a problem" tier is what makes it sustainable.

The second is a quarterly review. CI stops findings from accumulating, but it cannot stop the rules themselves from going stale. The model shutdown list needs updating, and the schedule definitions drift away from reality. Four times a year, I sit down and reconcile the MODEL_EOL dictionary and PERIOD_DAYS against how things actually run.

Adding prompts is easy. For an indie developer, if an idea occurs to you in the morning, you can have something working by the afternoon. Accumulate enough of them without a way to fold them back up, and eventually the billing statement is what tells you. The ledger is simply a way of going to get that news yourself, first.

Start by counting your call sites and recording last_hit. The classification logic can come later.