Check The Scoreboard, live at checkthescoreboard.com.
Josh Popkin built Check The Scoreboard after hearing the same question from real estate agent friends over and over: "Am I actually showing up when people ask ChatGPT who to hire?" Nobody had a real answer. Not because the agents weren't good at their jobs — because nobody was actually checking, consistently, in public, without an agenda attached to the answer.
The Problem Nobody Was Tracking
Ask ChatGPT who the best real estate agent in Brooklyn is, and it doesn't hand back ten blue links to sort through. It names about five people. Most people never look past that list. That's the entire ballgame now for a huge number of buyer and seller inquiries — and almost no agent knew where they stood on it, whether they were on it at all, or why the agent two blocks over kept getting named instead of them.
That gap is what Check The Scoreboard exists to close. It's a free tool, live now for New York City real estate agents, that asks the same 50 real questions people actually type when looking for an agent — questions like "Who are the best real estate agents in Park Slope?" or "Who should I hire to sell a brownstone in Brooklyn?" — every single day, and publishes exactly what ChatGPT says. Word for word. No edits, no reordering, no opinions layered on top.
Why Build a Public Tool Instead of Just Sharing Answers Privately?
Because a private answer isn't trustworthy, and Popkin wanted something agents could verify themselves rather than take on faith. So the site runs on three rules built in from day one — a deliberate response to the fact that Popkin also runs a company that gets paid to help businesses show up better in AI answers. That's a real conflict of interest, and rather than hide it, the rules are designed to neutralize it directly:
Nobody can pay to be listed. Not for any price, ever. The lists come from a program that publishes automatically — no person approves or adjusts what goes up. Clients of Popkin's company, Position Strategy Group, get nothing here: no placement, no edits, no early look at the data. If a paid feature is ever added to the site, it will say "sponsored" in large letters. The lists themselves never will.
The site publishes what the chatbot said, not what anyone thinks. The names, the order, the firm names, even ChatGPT's own one-line reasoning for each pick — all of it goes up exactly as given.
Every question is asked from a blank account, every time. No saved chat history, no memory, no location data shaping the answer — just a fresh, unbiased question asked the way a real prospective client would ask it.
What Happens When ChatGPT Gets It Wrong?
That gets published too. Sometimes the model names a firm when someone asked for a person, or repeats something out of date. That's not hidden — it's part of what makes the record useful. These lists aren't anyone's opinion of who's good at their job. They're a record of what a chatbot said on a given day, tracked daily so agents can watch how the answer moves over time instead of guessing based on one query.
What's Coming Next
Right now, the site tracks ChatGPT only, with Gemini and Perplexity on the way — future lists will state clearly which chatbot produced each one. The feature generating the most excitement internally is currently being tested: highly personalized scoreboards for individual agents, paired with specific, actionable feedback on why they aren't showing up higher. Not vague advice — actual diagnosis. Something like: "Everyone ranking above you has a completed profile on [X site] and yours is incomplete and unindexed. Here's exactly how to fix that, for free, today." The goal is turning a passive daily list into something an agent can act on immediately, without needing to hire anyone to explain it.
Why This Matters Beyond Real Estate
Popkin started with real estate agents because that's where the question came up first, and because "who should I hire" is exactly the kind of query where an AI-generated answer now matters more than a search results page ever did. But the underlying problem — nobody having a transparent, trustworthy way to see where they stand in AI-generated recommendations — isn't unique to one industry or one city. NYC real estate is where Check The Scoreboard starts. It isn't where it stops.
Check The Scoreboard is live and free at checkthescoreboard.com. Related reading: How Does Answer Engine Optimization (AEO) Work? and How We Increased AI Visibility for a Home Builder. More about Josh Popkin.