Question 1

What does the starter kit produce?

Accepted Answer

A complete workflow spec for a Claude-or-equivalent issue-triage agent: a label taxonomy, a duplicate-detection strategy (semantic embeddings vs exact title vs hybrid), confidence thresholds with explicit routing per band (auto-label, queue for maintainer review, defer), a webhook event handler spec, an escalation matrix, and stale-issue handling. Output is markdown plus a GitHub Actions YAML stub you can drop into `.github/workflows/`.

Question 2

Is the LLM allowed to close issues automatically?

Accepted Answer

By default, no. The kit's principle is human-in-the-loop: the agent proposes labels, suggests duplicates with links, writes a first-draft reply or needs-info prompt, and leaves the close-button click to a human. You can flip the auto-close switch for specific high-confidence cases (e.g. confidence > 0.95 and label = duplicate with a verified link) but it ships with the safe defaults.

Question 3

How does duplicate detection work?

Accepted Answer

Hybrid by default. First pass: exact title match (cheap, catches obvious copies). Second pass: vector embedding of issue body, search the last 1000 closed issues for cosine similarity > 0.85. If both hit, auto-label as `duplicate` with link. If only the vector pass hits, label as `possible-duplicate` and queue for human review. The kit defines the thresholds; you can tune them per repo.

Question 4

What are the confidence thresholds for?

Accepted Answer

Same pattern as the [AI Lead Scoring Pack](/ai-lead-scoring-pack/): the agent returns a tier and a confidence between 0 and 1. The kit defines auto, review, and defer bands. Above auto (default 0.85): labels applied, draft reply posted. Between review and auto: same actions but a maintainer is pinged. Below defer (default 0.4): no action, the issue waits for human triage. Three thresholds beat one floor every time.

Question 5

Which webhook events does it handle?

Accepted Answer

Minimum: `issues.opened` (the primary triage event) and `issue_comment.created` (for comment-based commands like `/needs-info` or `/duplicate-of`). Optional: `issues.edited` (re-triage if the body changed substantively), `issues.labeled` (cascade labels when a maintainer overrides). The spec includes idempotency keys so re-deliveries don't double-label.

GitHub Issue-Triage KI Starter-Kit GitHub Issue Triage AI Starter Kit

ErforderlichRequiredRepo-KontextRepo context

Label-TaxonomieLabel taxonomy

Duplikat-ErkennungDuplicate detection

Confidence-SchwellenConfidence thresholds

Webhook-EventsWebhook events

Eskalations-RegelnEscalation rules

Generierte SpecGenerated spec

Triage-Agent auf Ihren Repos in 3 WochenTriage agent on your repos in 3 weeks