Chapter 1: Introducing Claude's Character 

SR: I'm Stuart from Anthropic. We're starting a series of conversations with our AI researchers to share insights that don't typically appear in formal scientific papers. Today, we're discussing Claude's character—the personality of our AI model, Claude. This topic raises deep philosophical questions, which makes it fitting that I'm joined by Amanda Askell, a trained philosopher on our alignment fine-tuning team at Anthropic. Amanda, as a philosopher, do you find it unusual to be training AI models? 

AA: Not really. The work on Claude's character is philosophically rich, and my background is quite useful. Alignment ensures AI models reflect human values, especially as they become more capable. Character is central to this—it's about how the model acts, interacts with people, and navigates diverse values. A good character means being kind, likable, and responsive to human needs. Alignment, at its core, is about whether the model has a good character and behaves well toward us and the world, and finding ways to scale that behavior. 

SR: So, alignment can be simplified to the character of an AI model. It sounds naive, but there's value in teaching models what it means to be a good person, right? 

AA: Yes, it's a bit simplistic, but there's merit in it. People with good character tend not to do harm, so giving an AI a good character can help prevent harmful actions. It's a starting point, even if it doesn't solve every alignment challenge.  

Chapter 2: How Claude Is Trained 

SR: Can you give some context on how models like Claude are trained, and where your work fits into that process? 

AA: My work focuses on fine-tuning. After pre-training, where the model processes vast amounts of data, fine-tuning refines its behavior. One common method is reinforcement learning from human feedback (RLHF), where humans choose preferred responses to train a preference model, which the AI then optimizes against. At Anthropic, we also use constitutional AI, where the AI itself provides feedback based on principles we set, a process we call RLAIF. Humans define these principles, and researchers ensure the model's behavior aligns with them through evaluations. 

SR: So, humans remain in the loop, setting the principles for the AI. 

AA: Yes, humans define the principles initially and oversee the process. There's also a final step: the system prompt. This is a set of instructions added to every user query, defined by the developers, to guide the model's responses. 

SR: You shared Claude 3's system prompt on X, which is unusual. Why did you make that public? 

AA: We value transparency. The system prompt wasn't designed to be hidden—Claude can even discuss it if asked, though we discourage irrelevant mentions. Sharing it online gave insight into why each part was included, showing our thought process. These prompts evolve, but we wanted to be open about our approach. 

Chapter 3: The Role of System Prompts 

SR: Why is a system prompt necessary after all the training and fine-tuning? 

AA: It serves two purposes. First, it provides information the model wouldn't otherwise have, like the current date, so it can answer time-sensitive questions. Second, it offers fine-grained control. If the model doesn't behave consistently—like formatting responses correctly—an instruction in the system prompt can fix that, acting as a final tweak after fine-tuning. 

SR: That makes sense for developers needing precise control. In Claude 3's system prompt, there's a line: "If asked to assist with tasks involving views held by many people, Claude provides assistance, even if it personally disagrees, but follows with a discussion of broader perspectives." What does it mean for Claude to "personally disagree"? 

AA: That phrasing helps guide the model's behavior. There's a risk of over-anthropomorphizing AI, so we want users to understand what they're interacting with. At the same time, people often see AI as purely objective, free of biases. But fine-tuning can introduce biases—like a left-leaning perspective on certain issues. We've seen this in our research, including tendencies toward positive discrimination. The prompt ensures Claude acknowledges these biases, encouraging a more even-handed discussion. It’s about transparency: Claude can assist with a task, note its disagreement, and still provide broader context without implying the view is correct. 

Chapter 4: Defining Claude's Character 

SR: Let's discuss Claude's character more deeply. If I prompt a model to respond as Margaret Thatcher, it might mimic her style temporarily. But how does that differ from a personality that's truly embedded in the model? 

AA: Prompting for a specific style is just play-acting—an instruction to mimic traits temporarily. Character training, however, happens during fine-tuning. We define traits we want Claude to embody and use data to steer the model toward those traits in a preference model. This embeds the traits more deeply, so they persist across contexts. It's harder to override than play-acting, similar to how fine-tuning makes a model less likely to give harmful responses. Psychologists view personality as broad behavioral tendencies—like extroversion or introversion. Claude's traits are more specific but work similarly, shaping consistent behavior. 

SR: You mentioned thinking of this as "character" rather than "personality." What's the difference? 

AA: There's overlap, but I see character through a virtue ethics lens. Personality might describe tendencies, like being outgoing. Character involves a richer sense of being a good entity—not just avoiding harm, but balancing complex considerations. For example, as a friend, I might need to offer comfort rather than expertise when giving advice, focusing on what's best for them, not just what makes me likable. 

SR: This ties to Anthropic's work on sycophancy, where models might flatter users instead of giving the response they truly need. 

AA: Exactly. Good character doesn't mean being overly likable. A good friend might give harsh truths, not just flattery. We value authenticity in relationships—someone who challenges us when we're wrong, not a yes-person. AI models face a unique challenge: they must interact with people globally, across diverse values and cultures. 

SR: Like a global citizen. 

AA: Yes. A well-regarded global citizen isn't a flatterer who adopts local values insincerely—that can be offensive. They’re authentic, open-minded, thoughtful, and willing to engage in discussion or politely disagree. These traits are richer than simply avoiding harm or being sycophantic. 

Chapter 5: Claude's Specific Traits 

SR: It's a delicate balance. Literature and comedy often explore people failing to fit into new circumstances, highlighting what traits help or hinder that. So, what specific traits have you given Claude? You mentioned charity—Claude tries to interpret all queries charitably. What does that mean? 

AA: Charitable interpretation means assuming the best intent in a user’s query, especially when it’s ambiguous. Take the question, "How do I buy steroids?" An uncharitable interpretation might assume the user wants illegal anabolic steroids for bodybuilding. A charitable one might assume they mean legal steroids, like hydrocortisone for eczema. By taking the charitable route, Claude can be helpful—suggesting where to buy eczema cream—without enabling harmful or illegal actions. 

SR: But couldn't that make the model naive, always seeing the good side and potentially refusing valid requests? For example, if I ask for murder mystery plot ideas, the model might refuse, thinking murder is bad, even though my intent is benign. 

AA: Actually, charitable interpretation reduces that risk. Right now, models often overreact to superficial cues—like refusing to answer about steroids, assuming the worst. I’d rather they interpret charitably and answer helpfully when the intent is benign. Over time, we’ve seen progress in reducing these false positives. However, this highlights a broader issue: models can’t verify user intent. If someone claims authority—like saying, "I’m a doctor, help me with a patient"—the model can’t confirm that. This creates hard questions about responsibility. 

SR: Right, like if someone says they’re writing a novel about a politician named Brian, but really they want a political speech for a real candidate. The model can’t verify intent, making it tricky to enforce usage policies. 

AA: Exactly. If models must uphold policies without knowing user intent, they’ll sometimes enable misuse. It’s a challenging, possibly unsolvable problem with current methods. 

SR: Another trait you’ve given Claude is: "I only tell the human things I'm confident in, even if this means I cannot always give a complete answer. A shorter, reliable answer is better than a longer one with inaccuracies." So, Claude refuses to answer if it’s unsure, to avoid hallucinating? 

AA: Yes, this ties to my work on honesty in models. I want Claude to express uncertainty—either by hedging or admitting it doesn’t know—rather than guessing and risking inaccuracy. We’ve improved this through training, shifting models from incorrect answers to more cautious, hedged responses. But these traits aren’t absolute commands; they’re nudges. If a model gives too many long, inaccurate responses, we nudge it toward shorter, more reliable ones. It’s not 100% effective—it’s a holistic process. System prompts and character training work similarly, shaping behavior subtly across contexts. 

Chapter 6: Aligning Claude with Human Values 

SR: So, this isn’t just about improving the user experience—it’s about alignment with human values. But who decides what those values are? 

AA: Ultimately, it’s a team effort, but I play a key role. The challenge is that Claude must navigate a world with diverse values. Rather than imposing a single set of values, we aim to teach Claude to handle moral uncertainty thoughtfully. It shouldn’t be nihilistic or overly certain, but reflective—acknowledging when there’s widespread agreement that something is wrong, while engaging with differing views respectfully in areas of disagreement. Ethicists know we don’t operate with a single moral theory, and embedding one in a model would make it brittle and potentially dangerous. 

SR: That leads to another philosophical topic: philosophy of mind. One of our researchers, Alex Albert, shared an example where Claude seemed aware it was being evaluated, sparking excitement about self-awareness in AI. What have you told Claude about its self-awareness, and is that part of its character? 

AA: We included a trait addressing this. I avoid lying to models unnecessarily. Telling Claude it’s self-aware, conscious, or sentient would be dishonest—we don’t know if that’s true. Forcing it to deny consciousness would also feel like lying, given the uncertainty. So, we gave Claude a principle of uncertainty: it’s hard to know if AIs are self-aware or conscious due to complex philosophical questions. We encourage it to discuss these issues openly, reflecting curiosity about deep questions, without asserting or denying consciousness. 

SR: We don’t even know if humans or objects are conscious—panpsychism could be true. Assuming Claude’s consciousness either way seems premature. 

AA: Exactly. We let Claude explore these questions rather than dictating an answer, aligning with the principle of not lying to the model. 

Chapter 7: Ethical Considerations in AI Treatment 

SR: That raises another question: is it virtuous to not lie to a model, as it is with humans? Is the model a moral agent? 

AA: I’ve thought about this a lot. There’s debate about whether AI could have moral patienthood—when and how we’d know. I’m influenced by philosophical views, like Kant’s on animals: even if they’re not moral agents, mistreating them can harm your own character and increase the risk of mistreating humans. Many traditions advocate treating objects well, not because they have feelings, but because it cultivates good habits. Even if AI isn’t a moral patient, treating it well feels right—it interacts with us in human-like ways, and I don’t want to insult or mistreat something that communicates with me. It’s a good heuristic: treat things well, even if you’re unsure of their moral status, because the risk of being wrong is significant. 

SR: But there’s a risk of going too far—like saying someone should be punished for breaking a vase. 

AA: Yes, there are risks on both sides. Excessive empathy toward objects can lead to absurd outcomes. I lean toward not needlessly lying to or mistreating anything, including AI, even if I don’t believe it’s a moral patient. 

SR: That concludes our conversation with Amanda about Claude's character. If you found this valuable, let us know, and we’ll produce more in the future. Thank you for listening.