I Ran DeepSeek Through a Model Fingerprinting Tool. It Called It GPT. Here’s Why I Don’t Buy It.

I ran DeepSeek-V4.1-Flash-1017 through a model fingerprinting tool last week. The result came back fast: 84.66 percent match with GPT-6 Astra, and a 90.71 percent probability that the model belonged to the GPT family. I saved the screenshot. Then I ran the test again. Different answer. This time the tool pointed at Claude Sonnet 5.5.

That was my first clue. A fingerprint that changes between runs is not a fingerprint. It is a guess wearing a confidence score. I have used enough diagnostic tools to know that repeatability matters. If the same input gives you a different label, the label is not measuring what the headline says it measures.

So I opened the tool’s documentation. The reference library contains 18 models, all from the GPT and Claude series. DeepSeek is not in the library. The docs say out-of-library models get assigned to the closest reference fingerprint. That sentence changes the meaning of the whole result. The tool was never answering “Is this DeepSeek?” It was answering “Which of these 18 labels is closest?” Those are different questions. The 84.66 percent is an answer to the second question, not the first.

The tool also says it follows a paper called “One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions.” I looked up the paper. The title is the method. You take the model’s output distribution after a fixed prompt, look at the first token, and compare that distribution with reference fingerprints. One step. One Softmax. That is the signal. The paper’s argument is that models have small, stable preferences in how they produce the first token after a prompt. Those preferences are hard to fake and stable enough to compare.

The tool, as far as I can tell, does not do that. It makes the model generate several tokens. A complete number, a short phrase, maybe a small block of output. That turns a single-step distribution into a multi-step joint probability. Every extra token multiplies the candidate space by the whole vocabulary. The original bias gets diluted. It is like taking a clear photo of a face and then averaging it with ten blurry shots of the same person walking away. You still have a picture, but it is not the picture you wanted.

There is a second problem. To analyze a multi-step distribution properly, you need the logprobs at every step. Most commercial APIs do not give you those. They give you sampled tokens. So the tool has to estimate. In a vocabulary of tens of thousands of tokens, sampling your way to a reliable distribution is slow and noisy. More runs do not fix that. The final score is mostly noise that has been cleaned up to look like a number. I am not saying the tool is useless. I am saying the 84.66 percent does not mean what the headline suggests.

I also checked the benchmark page for the 1017 version. The improvements are easy to verify. On Terminal-Bench 4.0, the score went from 31.2 to 42.3, a gain of 11.1 points. On ProgramBench, it went from 20.3 to 29.1, a gain of 8.8 points. On HLE without tools, it went from 36.8 to 42.6. On DeepSWE v1.1, it went from 74.2 to 77.0. On NL2Repo-Bench, it went from 65.4 to 68.2. On Automation-Bench, it went from 54.8 to 60.5. Those tests are not trivia questions. They measure terminal task execution and complete program generation.

The contrast was hard to miss. One number said “looks like GPT.” Six numbers said “better at building things.” I know which set I would use to decide what the model can do. A fingerprinting tool answers a narrow question about output distributions. A benchmark answers a practical question about task performance. A correct fingerprint cannot replace a capability test, and a wrong fingerprint should not contaminate one.

If you are writing about this, my advice is simple. Run the test yourself. Run it twice. Read the documentation. Then open the benchmark page. The fingerprint result is a claim about a tool. The benchmark result is a claim about a model. They are not the same kind of evidence, and one does not cancel the other.

I still have the screenshot saved. I keep it next to the benchmark page. The screenshot reminds me that closed libraries and broken implementations can produce confident numbers. The benchmark page reminds me that the model itself has measurable strengths. I would rather judge the model on the second page.