Overview

Documentation at scale creates two persistent problems: you don’t know what’s missing until someone asks, and keeping docs in sync with code requires constant manual effort. I built two AI-powered tools to solve both: one that surfaces documentation gaps from real developer behavior, and one that closes the loop between code changes and documentation automatically.

Tool 1: Documentation Gap Analyzer

The Problem

Developers were asking the same questions in Slack that were already answered (or should have been answered) in documentation. There was no systematic way to know which questions indicated missing docs, which indicated discoverable docs, and which represented ROI opportunities for the documentation team to pursue.

The Solution

I built a pipeline that scrapes developer conversations and cross-references them against existing documentation to identify gaps, prioritize action items, and quantify the business impact of those gaps.

How it works:

1. Confluence Scraper: Pulls documentation content from Confluence spaces, normalizing page structure and content into a format suitable for LLM analysis
2. Slack Scraper: Extracts developer questions and conversations from relevant Slack channels, filtering noise and identifying question patterns
3. Gap Analysis Engine: Cross-references Slack questions against documentation corpus to determine whether each question is: answered by existing docs, partially answered, or completely unaddressed
4. LLM Scoring: Each gap is scored by an LLM for priority and severity based on question frequency, developer impact, and complexity of the missing information
5. Dual Report Output:
   • Action Items Report: Specific docs to write or update, ranked by priority/severity score, with direct references to the Slack questions driving each item
   • Executive ROI Report: Quantifies how much time developers spend answering questions in Slack that could be self-serviced, expressed in hours saved and engineering cost

Impact

• Gave the documentation team a data-driven prioritization framework instead of relying on gut feel or loudest stakeholder
• Made the business case for documentation investment in terms executives could act on
• Identified patterns of recurring questions that revealed systemic gaps in documentation coverage

Tools & Technologies

Python · Confluence API · Slack API · LLM API · Markdown · Data Analysis

Tool 2: PR-Driven Documentation Generator

The Problem

Code changes constantly. Documentation doesn’t keep up, not because writers don’t care, but because there’s no reliable signal that a given PR requires documentation changes. Writers can’t review every PR, and developers rarely know when or how to update docs alongside their code.

The Solution

I built an automated pipeline that monitors all pull requests, analyzes code changes against existing documentation, determines whether doc updates are needed, and opens a draft PR with generated documentation for human review.

How it works:

1. PR Analysis: On every pull request, the tool reads the diff and cross-references it against the documentation corpus to determine whether the changes require documentation additions, deletions, or updates
2. Impact Classification: Classifies each PR as: no doc changes needed, doc update recommended, or doc update required (with reasoning attached)
3. Doc Generation: For PRs requiring documentation changes, generates draft content using a custom skill I wrote that encodes my organization’s style guide, Diátaxis framework conventions, terminology standards, and structural patterns
4. Draft PR Creation: Opens a draft pull request with the generated documentation changes, including:
   • The generated content
   • Reasoning for why each change was recommended
   • References to the triggering code changes
   • A checklist for the human reviewer
5. Human Review Gate: All generated docs go through human review before merge. The tool accelerates the workflow; it doesn’t replace editorial judgment.

The Documentation Skill

A key component of this tool is the custom skill I wrote to govern how documentation is generated. The skill encodes:

• Style guide rules (tone, voice, terminology)
• Diátaxis content type conventions (tutorial vs. how-to vs. reference vs. explanation)
• Structural templates for common doc types
• Organizational conventions for formatting, linking, and labeling

This ensures generated docs are consistent with existing documentation and require minimal editing rather than a full rewrite.

Impact

• Reduced documentation lag: PRs no longer ship without a documentation signal attached
• Surfaced doc requirements earlier in the review process, when context is freshest
• Generated first drafts that writers could edit rather than write from scratch
• Created an auditable trail of why documentation changes were recommended for each PR

Tools & Technologies

Python · GitHub API · GitHub Actions · Claude Code · LLM API · Custom Documentation Skill · Diátaxis · CI/CD

Reflection

Both tools share a common design philosophy: use AI to surface signal, not to replace judgment. The gap analyzer tells you what to work on and why, but a human decides what to actually write. The PR generator opens a draft and explains its reasoning, but a human decides what ships.

The hardest part of both projects wasn’t the AI integration. It was defining what “good” looked like well enough to encode it. That meant writing the scoring rubric for gap severity, building the documentation skill with enough specificity to produce useful output, and designing reports that gave stakeholders the right information in the right format.

That work is fundamentally documentation work. The AI just made it scale.