P1AI Support Agent
2026Classifies Apple support tweets, drafts grounded replies, and decides when a human should take over. Then I graded my own grader.
I graded the grader, and it failed.
I scored 10 generated replies myself, blind, and compared them with the LLM judge's scores. The judge was lenient and couldn't rank quality, so its scores are reported as unvalidated instead of being passed off as results.
Also documented: 5[1] failure modes and 15[3] non-obvious design decisions.

