Build an Auto-Documentation Pipeline with Gemini API and GitHub Actions

Right after writing a function, you have the energy to document it properly. The problem shows up three months later — the function signature changed, a new parameter was added, and the docstring still describes the old version. Nobody's sure if the docs are trustworthy anymore, so everyone just reads the code directly. At that point, the docs become an active liability.

For solo developers especially, this pattern repeats endlessly. You push a feature, ship it, move on, and six months later you're the confused reader trying to understand your past self's decisions.

The pipeline I'll walk through here won't write your documentation for you. What it will do is interrupt the cycle: every time you open a Pull Request with Python changes, Gemini API scans the diff and posts a comment if any docstrings look out of sync. It's a nudge, not a replacement.

Why Documentation Keeps Getting Stale

The core issue is structural. Code changes are enforced by CI — tests fail, types fail, linting fails, the PR gets blocked. Documentation freshness doesn't have that enforcement. It's voluntary, and voluntary things get skipped when deadlines hit.

For personal projects, this creates a specific problem: you're the only person who can flag outdated docs, and you're also the one too busy to update them. Automating the reminder — even imperfectly — breaks that loop.

How the Pipeline Works

The approach is intentionally simple:

A Pull Request triggers a GitHub Actions workflow
git diff identifies which Python files changed
The diff and current file content get sent to Gemini 2.5 Flash
Gemini checks whether docstrings still match the code
If updates are needed, a PR comment is posted with specific suggestions

The key design decision: Gemini suggests, humans decide. The pipeline never modifies files directly. This keeps a human in the loop for every change, which matters when AI-generated documentation can be plausible but subtly wrong.

For related patterns, Gemini API with GitHub Actions for code quality covers a complementary approach focused on code correctness rather than documentation.

Step 1: The GitHub Actions Workflow

Create .github/workflows/doc-review.yml in your repository:

# .github/workflows/doc-review.yml
name: AI Document Review
 
on:
  pull_request:
    types: [opened, synchronize]
    paths:
      - '**/*.py'   # Only fires when Python files change
 
jobs:
  doc-review:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write   # Required to post PR comments
 
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0   # Full history needed to compute diffs
 
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
 
      - name: Install dependencies
        run: pip install google-genai PyGithub
 
      - name: Run AI doc review
        env:
          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
          REPO_NAME: ${{ github.repository }}
          BASE_SHA: ${{ github.event.pull_request.base.sha }}
          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
        run: python scripts/doc_reviewer.py

Add your GEMINI_API_KEY under Settings → Secrets and variables → Actions. You can get a free API key from Google AI Studio.

Step 2: The Review Script

Create scripts/doc_reviewer.py:

# scripts/doc_reviewer.py
import os
import subprocess
from github import Github
import google.genai as genai
 
GEMINI_API_KEY = os.environ["GEMINI_API_KEY"]
GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
PR_NUMBER = int(os.environ["PR_NUMBER"])
REPO_NAME = os.environ["REPO_NAME"]
BASE_SHA = os.environ["BASE_SHA"]
HEAD_SHA = os.environ["HEAD_SHA"]
 
# Flash over Pro: roughly 10x cheaper and accurate enough for doc review tasks.
# For production code review you might prefer Pro, but for high-volume CI use,
# Flash keeps costs negligible.
MODEL_ID = "gemini-2.5-flash"
 
def get_changed_python_files() -> list[str]:
    """Return Python files modified in this PR."""
    result = subprocess.run(
        ["git", "diff", "--name-only", f"{BASE_SHA}...{HEAD_SHA}"],
        capture_output=True, text=True, check=True
    )
    return [f for f in result.stdout.strip().split("\n") if f.endswith(".py")]
 
def get_file_diff(filepath: str) -> str:
    """Get the unified diff for a file with 5 lines of context."""
    result = subprocess.run(
        ["git", "diff", "-U5", f"{BASE_SHA}...{HEAD_SHA}", "--", filepath],
        capture_output=True, text=True, check=True
    )
    return result.stdout
 
def get_current_content(filepath: str) -> str:
    """Read the current file content. Returns empty string if deleted."""
    try:
        with open(filepath, "r", encoding="utf-8") as f:
            return f.read()
    except FileNotFoundError:
        return ""
 
def review_with_gemini(filepath: str, diff: str, content: str) -> str | None:
    """
    Ask Gemini to review whether docstrings match the changed code.
 
    Returns a suggestion string if updates are needed, None otherwise.
    """
    client = genai.Client(api_key=GEMINI_API_KEY)
 
    prompt = f"""Review the following Python file changes and current code.
Check whether the docstrings (function, class, or module descriptions)
are still accurate after the changes.
 
If docstrings need updating, provide specific suggestions in Markdown.
If everything looks fine, respond only with: "No documentation updates needed."
 
## File
{filepath}
 
## Diff
```diff
{diff[:3000]}

Current Code

{content[:5000]}

Output Format (only if updates needed)

Name of function or class that needs updating
What's wrong with the current docstring (1-2 sentences)
Suggested updated docstring (in a Python code block) """

response = client.models.generate_content( model=MODEL_ID, contents=prompt, )

suggestion = response.text.strip()

Skip "no updates needed" responses to keep PR comments meaningful

if "no documentation updates needed" in suggestion.lower() or len(suggestion) < 30: return None

return suggestion

def post_pr_comment(suggestions: list[tuple[str, str]]) -> None: """Post all suggestions as a single PR comment.""" if not suggestions: print("No suggestions — skipping PR comment") return

gh = Github(GITHUB_TOKEN)
repo = gh.get_repo(REPO_NAME)
pr = repo.get_pull(PR_NUMBER)

body = "## 🤖 AI Documentation Review (Gemini)\n\n"
body += "The following documentation updates may be worth reviewing "
body += "based on the code changes in this PR.\n"
body += "_Auto-generated suggestions — please review before applying._\n\n---\n\n"

for filepath, suggestion in suggestions:
    body += f"### 📄 `{filepath}`\n\n{suggestion}\n\n---\n\n"

pr.create_issue_comment(body)
print(f"✅ Posted {len(suggestions)} suggestion(s) to PR")

def main(): changed_files = get_changed_python_files()

if not changed_files:
    print("No Python files changed — nothing to review")
    return

print(f"📂 Reviewing {len(changed_files)} file(s)")
suggestions = []

# Cap at 10 files to avoid cost spikes on large refactors
for filepath in changed_files[:10]:
    print(f"🔍 Reviewing: {filepath}")
    diff = get_file_diff(filepath)
    content = get_current_content(filepath)

    if not diff:
        continue

    suggestion = review_with_gemini(filepath, diff, content)
    if suggestion:
        suggestions.append((filepath, suggestion))
        print(f"  → Suggestion generated")
    else:
        print(f"  → No update needed")

post_pr_comment(suggestions)

if name == "main": main()


## What Works Well and What Doesn't

After running this on a live project for two weeks, here's an honest assessment.

**Where it reliably adds value:** When a function gains a new parameter and the `Args:` section isn't updated, Gemini catches it consistently. Same with `Raises:` when new exceptions are introduced. These are the easy misses in manual review, and the pipeline catches them every time.

**Where it generates noise:** Functions that never had docstrings in the first place get flagged on every PR. That's technically correct, but it can feel repetitive. If you want to reduce this, add a clause to the prompt like "skip functions that have no existing docstring" — though you'll lose the reminder to add them in the first place.

Another common false positive: refactors that rename variables internally without changing behavior. The diff looks significant to the model, but the docstring is still accurate. Adding more context about what the refactor does (via PR description) can help, though the model doesn't currently read that.

For a more comprehensive automated code review approach, building an AI PR review bot with Gemini covers the full pattern including logic and security review.

## Cost Estimates

Using Gemini 2.5 Flash, each PR typically consumes between 3,000 and 8,000 tokens depending on the size of the diff. At that rate:

- **Free tier** (15 requests/day): sufficient for solo projects with modest PR volume
- **Pay-as-you-go**: processing 100 PRs per month costs a few cents, not dollars

The 10-file cap is important. Without it, a large refactor touching 50 files could blow up the token count significantly. If you want to handle larger PRs, consider prioritizing files by diff size rather than just taking the first 10.

## Extending the Pipeline

A few common customizations worth knowing before you commit to a specific design.

**Filtering by diff size:** Rather than always taking the first 10 files alphabetically, rank by diff length so the most-changed files get reviewed first:

```python
def get_changed_python_files_by_size() -> list[str]:
    result = subprocess.run(
        ["git", "diff", "--numstat", f"{BASE_SHA}...{HEAD_SHA}"],
        capture_output=True, text=True, check=True
    )
    files = []
    for line in result.stdout.splitlines():
        parts = line.split("\t")
        if len(parts) == 3 and parts[2].endswith(".py"):
            added = int(parts[0]) if parts[0].isdigit() else 0
            deleted = int(parts[1]) if parts[1].isdigit() else 0
            files.append((added + deleted, parts[2]))
    return [f for _, f in sorted(files, reverse=True)][:10]

Skipping generated files: Auto-generated Python files (Django migrations, protobuf stubs, etc.) produce noisy suggestions. Filter them out:

SKIP_PATTERNS = ["migrations/", "_pb2.py", "generated/", "test_"]
 
def should_skip(filepath: str) -> bool:
    return any(pattern in filepath for pattern in SKIP_PATTERNS)

Then in main(), add if should_skip(filepath): continue before calling review_with_gemini.

Upgrading to Gemini 2.5 Pro: For projects where documentation precision matters most, swapping the model constant takes one line. Flash handles the common cases well, but Pro tends to produce more accurate Args: and Returns: sections when the function logic is complex.

Getting Started

Add the workflow file and the script to an existing repository and open a test PR with a Python change. The first run will show you exactly what kinds of suggestions Gemini produces for your codebase — which also helps you calibrate whether to tighten or loosen the prompt.

The value isn't in generating perfect docs automatically. It's in making "I should check if this docstring is still accurate" a question that gets asked automatically, rather than one that depends entirely on whoever happens to remember.