The central thesis

The report argues that standard performance benchmarks create what researchers call "super spiky capability distributions." Models get optimized to solve hard coding puzzles, but this does not mean they write good code, that is, code that is secure, reliable, and maintainable over time. Researchers from SonarSource used their static analysis engine to look beyond whether code works to examine how it is written.

The shared foundation

Before distinguishing personalities, the report establishes what all models have in common. These are the capabilities that made AI coding assistants so popular.

High syntactic competence. Syntactic competence emerges because code syntax is highly structured and repetitive. During training, models process billions of code examples and learn statistical patterns. The correct placement of semicolons, brackets, and keywords appears in virtually every training example. Models learn these patterns so thoroughly that syntax errors become rare. This is analogous to how a person reading thousands of English sentences would eventually internalize grammar rules without explicit instruction.

Algorithmic problem-solving reflects the abundance of algorithm implementations in training data. GitHub, Stack Overflow, textbooks, and documentation contain countless examples of sorting algorithms, search procedures, graph traversals, and similar patterns. When asked to implement a binary search, the model has seen hundreds of variations and can synthesize a working version. The models are essentially sophisticated pattern matchers with vast libraries of examples to draw from.

Cross-language translation works because algorithms are abstract concepts that exist independently of any particular language. A quicksort in Python follows the same logical structure as a quicksort in Java. The model learns to recognize the underlying pattern (divide, partition, recurse) and can express it using the vocabulary of different languages. This is similar to how a bilingual person can explain the same idea in two languages; the concept exists separately from its expression.

However, these strengths have a common limitation. They all depend on pattern matching against training data. The models excel when the problem resembles something they have seen before. They struggle when tasks require tracking state across an entire program (like preventing injection attacks), maintaining consistency over time (like proper resource cleanup), or reasoning about interactions between distant parts of code (like concurrency bugs).

The models also share profound weaknesses. All of them struggle to write secure code because security requires tracking data flows across an entire application, something that exceeds their context window. They all generate hard-coded secrets because such flaws exist in their training data. And perhaps most telling, code smells (maintainability issues) represent roughly 90 to 95 percent of all issues across every model. This suggests a structural tendency toward producing code that will become difficult to modify, extend, or debug over time.

The personality dimensions

The researchers measured three traits that distinguish each model's style.

Verbosity captures sheer volume. Given identical tasks, GPT-5-minimal generated about 490,000 lines of code while OpenCoder-8B produced only 120,000. This is not just length for length's sake. A verbose model attempts comprehensive, self-contained solutions. A concise model aims for the quickest working path.

Complexity measures structural and logical intricacy. GPT-5-minimal scored over 111,000 on cognitive complexity, more than twelve times higher than OpenCoder-8B's 14,000. High complexity suggests a model that builds elaborate, multi-layered architectures. Low complexity indicates linear, straightforward thinking.

Communication style shows itself through comment density. Claude 3.7 Sonnet commented 16.4 percent of its code, nearly three times more than Claude Sonnet 4 and eight times more than GPT-5-minimal at just 2.1 percent. Some models explain their work; others assume their logic speaks for itself.

The models tested

The study examined six models representing different approaches to AI-assisted coding.

Claude Sonnet 4 was released by Anthropic on May 22, 2025. Anthropic does not publicly disclose parameter counts for its models. It features a 200,000 token context window. In the study, it achieved the highest functional performance at 77.04% benchmark pass rate, with the lowest issue density at 19.48 problems per thousand lines of code.

Claude 3.7 Sonnet was released by Anthropic on February 24, 2025. It introduced hybrid reasoning capabilities, allowing users to choose between fast responses and step-by-step reasoning. It passed 72.46% of benchmarks and stood out for its exceptional documentation habits.

GPT-4o was released by OpenAI on May 13, 2024. The "o" stands for "omni" and reflects its multimodal capabilities across text, audio, and images. OpenAI does not officially confirm parameter counts, but industry estimates suggest approximately 1.8 trillion parameters. It uses a 128,000 token context window and achieved a 69.67% pass rate in the study.

GPT-5-minimal refers to OpenAI's GPT-5 model running in minimal reasoning mode. GPT-5 was released on August 7, 2025 and operates as a system of multiple sub-models with an intelligent router that selects the appropriate variant based on task complexity. The "minimal" mode tested by Sonar corresponds to the fastest, least resource-intensive configuration. It achieved 75.37% on benchmarks but generated the most code and highest complexity of any model.

Llama 3.2 90B was released by Meta on September 25, 2024. It contains approximately 90 billion parameters and supports multimodal processing of text and images with a 128,000 token context window. Unlike the commercial models, Llama is available under an open license. It achieved only 61.47% on benchmarks and exhibited the worst security profile of any model tested.

OpenCoder-8B was developed by INF Technology (Shanghai) Co., Ltd. and released in November 2024. Unlike the proprietary models, OpenCoder is fully open-source. The developers published the complete data-cleaning pipeline, training protocols, and 4.5 million supervised fine-tuning examples. The model was trained from scratch on 2.5 trillion tokens. The "8B" refers to approximately 8 billion parameters, making it dramatically smaller than its peers. To run OpenCoder-8B locally requires roughly 5-16 GB of memory depending on precision settings, whereas a 90-billion parameter model needs specialized hardware with 50+ GB. This accessibility comes at a cost: it achieved the lowest benchmark pass rate at 60.43% and the highest issue density at 32.45 problems per thousand lines.

The six archetypes

With specifications established, we can examine how each model's personality manifests in practice.

Claude Sonnet 4: the senior architect. This model codes like an ambitious senior engineer building enterprise-grade systems. It achieved the highest functional performance at 77 percent, and it consistently attempts to implement sophisticated safeguards, error handling, and advanced features. But this very sophistication creates a trap. Teams may feel safer because the code looks advanced, while in reality it introduces complex, high-severity bugs. Notably, 9.81 percent of its bugs involve concurrency and threading issues, and 15.07 percent involve resource leaks. Its strength and weakness are the same thing: ambition.

Claude 3.7 Sonnet: the balanced predecessor. This earlier Anthropic model passed 72.5 percent of benchmarks and stands out as an exceptional documentarian with that remarkable 16.4 percent comment density, nearly three times higher than its successor and the highest of any model evaluated. Its code is uniquely readable. But the researchers warn against assuming it is safer just because it appears less reckless. It still produces 56 percent BLOCKER*-severity vulnerabilities and shares the foundational flaws of all models.

GPT-5-minimal: the baseline performer. OpenAI's reasoning model in its minimal reasoning mode delivered 75 percent pass rates with strong performance. But it generated the most code (490,010 lines) and the highest complexity scores (111,133 cognitive complexity). Its personality shows a more traditional risk profile with common, well-understood flaws like path-traversal vulnerabilities. However, its verbosity leads to the highest proportion of CRITICAL*-severity code smells, meaning the code will become increasingly difficult to maintain over time.

GPT-4o: the efficient generalist. This model is the jack-of-all-trades at 70 percent pass rate. It sits in the middle on most metrics, neither the most verbose nor the most concise, neither the most complex nor the simplest. Its distinctive trait appears in what goes wrong: 48 percent of its bugs are control-flow mistakes, the highest proportion of any model. It grasps the main objective but fumbles the details. The code works for the intended scenario but accumulates persistent edge-case problems.

Llama 3.2 90B: the unfulfilled promise. Given its scale and Meta's backing, this model should be a top contender. Instead, it passed only 61.5 percent of benchmarks, barely better than the much smaller OpenCoder-8B. More alarming, 70.73 percent of its vulnerabilities are BLOCKER severity, the worst security profile of any model tested. This means that when Llama introduces security flaws, they tend to be the kind that could directly enable attacks or system failures, not minor weaknesses that might theoretically be exploited under unusual conditions. The researchers suggest deploying it in production without aggressive external verification carries substantial risk.

OpenCoder-8B: the rapid prototyper. This small open-source model is the brilliant but undisciplined junior developer. It produces the least code and achieved the lowest pass rate at 60 percent. But its issue density of 32.45 problems per thousand lines is the highest of any model, and 43 percent of its code smells are dead, unused, or redundant code. This is the classic sign of rushed, iterative development without cleanup. Perfect for hackathons and proofs-of-concept, but its code would need significant refactoring before production.

The troubling discovery about progress

Perhaps the most counterintuitive finding concerns what happens when models become more capable. Comparing Claude 3.7 Sonnet to Claude Sonnet 4 reveals a 6.3 percent improvement on benchmarks but a marked increase in the severity of mistakes. The newer model's bugs and vulnerabilities are more likely to be BLOCKER severity.

GPT-5-minimal demonstrates an even more complex trade-off. While it reduces common BLOCKER vulnerabilities dramatically, it introduces a new class of risk. Its concurrency and threading bugs jumped to 20 percent of all bugs, compared to 1.44 percent for Claude 3.7 Sonnet. As models attempt more sophisticated solutions, they generate more sophisticated problems.

The researchers describe this as a shift from common, well-understood flaws to subtle, harder-to-detect implementation challenges. The issues in lower-reasoning models are often easier to spot because they are more straightforward. More capable models hide their problems better.

Why this matters

What Sonar has revealed is that these models are not neutral code generators. They are opinionated authors with distinct styles, tendencies, and failure modes. Just as a thoughtful editor learns which writers produce clean first drafts versus which ones need heavy revision, teams adopting AI coding assistants must learn the personality of their chosen model.

The Claude Sonnet 4 personality requires reviewers who understand concurrency, resource management, and the subtle bugs that hide in sophisticated code. The GPT-4o personality requires attention to edge cases and control-flow logic. The OpenCoder-8B personality requires aggressive cleanup passes before anything goes to production.

This is the kind of nuanced understanding that standard benchmarks never provide. And as these models become more autonomous, writing more code with less human oversight, understanding their characteristic ways of thinking becomes not just useful but essential.

The research presented by  SonarSource, a Swiss company that has spent over 16 years building static analysis tools for detecting bugs, vulnerabilities, and code quality problems in enterprise software. Their flagship product, SonarQube, is widely used in professional development environments to automatically scan code and flag issues before they reach production. This background matters because it means the researchers brought a mature, battle-tested framework for evaluating code quality rather than inventing new metrics for this study.

The report, titled "The Coding Personalities of Leading LLMs," was published in October 2025. SonarSource developed a proprietary analysis framework specifically for assessing AI-generated code, combining their static analysis engine with established practices from coding model evaluations. They tested six models on over 4,442 Java programming assignments drawn from recognized benchmark sources including MultiPL-E and ComplexCodeEval. The goal was to move beyond the standard performance benchmarks that dominate AI model comparisons and examine what the generated code actually looks like: whether it is secure, whether it follows engineering best practices, and whether it will be maintainable over time. The researchers explicitly state that their findings are intended both for model developers seeking concrete improvement targets and for software teams trying to choose the right AI assistant for their particular needs.