Let us test this prompt helper and analyzer tool
Artificial Intelligence

Let us test this prompt helper and analyzer tool

I just sendt this to Opencode with Superpower installed

Jan Petersen
April 17, 2026
5 min read
1 views

The start prompt

/brainstorming i like to build a app that will install Copilot hock in a repo that the user points to and, it should make hock for all the states that are in Github copilot-cli, meaning i like to save all prompts and i like to save them i a Github repo under a Issue, so all prompts will be saved here in the repo but as comment to first isus, by that i will have all prompts made in the copilot-cli doing the vibe coding, under the same issue The app should the have a other function that is user points to a Github repo (maybe the one above) but any and also the issue, then the app should analyze the issue incl all the comments, and give a summary og how good are the prompts overall and on every single prompt, with a carector from 1 to 5 and a short comment on what could improve it and what is good
Build MiniMax M2.5 Free OpenCode Zen

It was good in asking questions! and i added information and needs until the plan was written, this is very good i am convinced that the better the plan is from start the faster and less fails you will have.

The plan it came back with (i will make a Post on how it ended up with, even if its go wrong)

I might try this also in Github copilot-cli with Superpower installed


Prompt Helper and Analyzer - Implementation Plan

Overview

A TypeScript/Node tool that captures Copilot CLI prompts via hooks and provides a web dashboard to analyze prompt quality using Copilot SDK.

Architecture

Components

  1. CLI Tool - Generates hook configs, manages GitHub integration
  2. Copilot CLI Hooks - Captures prompts via .github/hooks/copilot-journal.json
  3. Web Dashboard - Uses Copilot SDK for prompt analysis and history viewing

Data Flow

User types prompt in Copilot CLI
    ↓
userPromptSubmitted hook fires
    ↓
Hook script posts prompt as GitHub issue comment
    ↓
Dashboard fetches issue + comments → analyzes with Copilot SDK

Commands

prompt-helper init [--issue-title "My Journal"]

  1. Check gh authentication
  2. Create GitHub issue in current repo
  3. Generate .github/hooks/copilot-journal.json
  4. Create hook scripts in .github/hooks/scripts/

prompt-helper analyze <issue-url> [--model gpt-5.1]

  • Fetch issue and all comments
  • Use Copilot SDK to analyze each prompt
  • Output scores and feedback

prompt-helper models

  • Fetch available models from Copilot CLI
  • Display to user

prompt-helper dashboard [--port 3000]

  • Start web server
  • Serve dashboard UI

Configuration

.prompt-helper/config.json

json
{
  "defaultModel": "gpt-5.1",
  "scoringWeights": {
    "clarity": 40,
    "context": 30,
    "completeness": 30
  },
  "github": {
    "tokenEnvVar": "GH_TOKEN"
  }
}

.prompt-helper/repos.json

json
[
  {
    "repo": "owner/project",
    "issueNumber": 1,
    "issueTitle": "Copilot Journal - 2026-04-17"
  }
]

Scoring Criteria (Configurable)

DimensionWeight (default)Description
Clarity40%Is the intent clear? Specific files mentioned? Desired outcome stated?
Context30%Relevant background? Constraints? Scope appropriate?
Completeness30%All details included? Edge cases considered? Acceptance criteria clear?

Analysis Output (per prompt)

  • Score: 1-5 stars
  • What's good ✓
  • What could improve ✗
  • Overall weighted average

Project Structure

prompt-helper/
├── src/
│   ├── cli/
│   │   ├── commands/
│   │   │   ├── init.ts
│   │   │   ├── analyze.ts
│   │   │   ├── models.ts
│   │   │   └── dashboard.ts
│   │   └── index.ts
│   ├── hooks/
│   │   └── capture-prompt.ts
│   ├── lib/
│   │   ├── github.ts
│   │   ├── config.ts
│   │   └── analyzer.ts
│   └── dashboard/
│       ├── server.ts
│       └── public/
├── package.json
├── tsconfig.json
└── README.md

Hook Files Structure

<repo>/.github/hooks/
├── copilot-journal.json    # Hook config
└── scripts/
    ├── capture-prompt.sh   # Bash (Linux/Mac)
    └── capture-prompt.ps1  # PowerShell (Windows)

Dashboard Features

Home Page

  • List tracked repos and issues
  • Stats: total prompts, average score, sessions count

Issue Detail

  • All prompts from issue as comments
  • Each prompt: timestamp, cwd, full text
  • "Analyze" button per prompt

Analysis Results

  • Overall score (weighted average)
  • Per-prompt scores with feedback

Export

  • Download as Markdown
  • Download as JSON

Settings

  • Model selector (from Copilot CLI)
  • Sliders for scoring weights (must sum to 100%)

Tech Stack

  • TypeScript
  • Node.js
  • Express.js or Fastify
  • @github/copilot-sdk
  • GitHub API (via gh CLI or octokit)

Implementation Tasks

Phase 1: CLI Core

  1. Initialize project with TypeScript
  2. Set up CLI framework (commander or yargs)
  3. Implement config management
  4. Implement GitHub API integration (gh CLI)

Phase 2: Hook System

  1. Create init command
  2. Generate hook JSON config
  3. Create hook scripts (bash + PowerShell)
  4. Test hook integration

Phase 3: Analysis

  1. Implement models command (fetch available models)
  2. Integrate Copilot SDK
  3. Implement analyze command
  4. Implement scoring logic

Phase 4: Dashboard

  1. Set up Express/Fastify server
  2. Create dashboard UI
  3. Implement issue fetching
  4. Implement analysis in dashboard
  5. Add export functionality
  6. Add settings page

Phase 5: Polish

  1. Add error handling
  2. Write tests
  3. Build and package
  4. Documentation

Share this post

Related Posts

Comments

Be the first to leave a comment.

Leave a comment

Comments are reviewed before they appear.