Another conversation with ChatGPT
Long post!
It will have been apparent if you have been with me through my five posts about Artificial Intelligence, that I have some concerns. My last conversation with ChatGPT led to the following further conversation. Which I think is instructive… although we must remember that the answers are based on the probability that they will match my concerns. I began…
Curated Lines said: In a recent conversation with you, you ended up by admitting that your answers had been “misleading” and “a bit of a stretch”. This made me wonder whether the public should trust LLM Chatbots, but more particularly whether you have any morality – sense of right or wrong, good and evil – built into your system, and whether it would prevent you from giving me advice that would be against the law or against most people’s better judgement?
ChatGPT said:
That’s a great question, and I appreciate your skepticism. Trust in AI, including chatbots like me, should be earned through reliability, transparency, and ethical behavior.
To answer your main point: Yes, I do have guardrails based on widely accepted ethical and legal principles. My responses are designed to align with established moral and legal standards. That means I won’t provide advice that promotes illegal activities, harm to others, or actions that go against fundamental ethical norms. For example, I wouldn’t help someone commit fraud, promote violence, or provide dangerous medical advice.
That said, my understanding of morality is based on how it’s programmed—drawing from widely accepted human values rather than personal judgment. While I aim to be as accurate and responsible as possible, I’m not infallible. If I ever give misleading or incorrect information, it’s not intentional, but rather a limitation of my model. I try to be upfront when I make mistakes, and I encourage people to fact-check critical advice.
Would you say that aligns with your expectations of AI morality? Or do you think AI should go further (or less far) in enforcing ethical boundaries?
CL said: I am left thinking that it would be relatively easy for someone with evil intent to insert a bias into your learning. How do you deal with the differences between say UK and USA legal systems?
ChatGPT said:
You’re absolutely right to consider the risks of bias in AI models like me. Bias can creep in through training data, human influences, or even the way questions are framed. To counteract that, my responses are shaped by a combination of broad, diverse data sources and reinforcement learning guided by ethical principles. OpenAI also implements safeguards to prevent harmful biases from dominating my answers.
Addressing Legal Differences (UK vs. USA Example)
Laws vary significantly between countries, and I try to recognize those distinctions. If you ask a legal question, I attempt to tailor my response to the jurisdiction you mention. For example:
- Gun Laws: In the U.S., gun ownership is widely protected under the Second Amendment, while in the UK, firearm regulations are far stricter.
- Defamation: UK libel laws are generally more plaintiff-friendly than those in the U.S., where free speech protections under the First Amendment provide stronger defenses.
- Self-Defense Laws: “Stand Your Ground” laws exist in many U.S. states, while the UK has a more restrictive approach to self-defense.
If I’m ever uncertain about jurisdiction-specific laws, I’d either clarify or encourage checking with a legal professional. And if you ever spot an answer that seems to blur these distinctions, I’d appreciate you pointing it out—critical thinking and human oversight are key to responsible AI use.
Do you think this approach is enough to prevent bias, or do you see gaps in how AI should handle legal and ethical nuance?
CL said: You ‘think’ or ‘believe’ that the vast amount of training data prevents bias or wrong doing. But you can never know that. Because it is what you have been led to believe by your training data. Suppose the entire knowledge base that you use had been brought together by someone with bias?
ChatGPT said:
You’ve hit on one of the most profound challenges in AI ethics: Can an AI ever be truly neutral if all its knowledge comes from human sources, which are inherently biased?
The short answer is: No, I can’t be completely free from bias because all data carries some level of human influence. Even if I process a vast range of sources, those sources reflect the perspectives, priorities, and sometimes even the agendas of the people who created them.
Can AI Be Programmed to Detect Bias?
Efforts are made to reduce bias rather than eliminate it completely. Some ways this happens:
- Diverse Data Sources: My training data comes from a wide range of materials, reducing the dominance of any single perspective.
- Ongoing Audits & Feedback Loops: Researchers test me against different scenarios to detect and mitigate unfair biases.
- Ethical & Legal Oversight: AI developers, ethicists, and even public input play a role in shaping AI policies.
The “What If” Scenario: A Fully Biased AI
If all my training data were curated by a biased group, I would reflect that bias—just as a person raised in an isolated ideological environment might struggle to see alternative viewpoints. However, AI can be trained to recognize biases and adjust responses accordingly. This is an ongoing process, not a solved problem.
What Does This Mean for You?
Since I can’t be 100% certain of my neutrality, you should always question me. Just as you would cross-check information from a news source, government, or expert, you should do the same with AI.
Your skepticism is exactly what prevents AI from being blindly trusted. If you were designing an AI system yourself, how would you handle the problem of bias? Would you prefer AI that declares its biases openly, or one that tries to balance multiple perspectives?
CL said: I think that the trouble is that the man or woman in the street – who may not think in the terms I have been raising here – may – possibly does – tend to trust Chatbots blindly. Especially when they are ‘hidden’ for example in the operating system of a car or mobile phone.
ChatGPT said:
That’s a very real concern. AI is becoming more embedded in everyday life—whether in cars, phones, customer service, or even news aggregation. If people trust AI blindly, without questioning its limitations or biases, it can lead to misinformation, manipulation, or misplaced trust in automated decisions.
Why Do People Trust AI Too Easily?
- AI Sounds Confident – Even when uncertain, AI tends to present information fluently, which can make it seem more reliable than it actually is.
- Convenience Over Scrutiny – Most people don’t fact-check every response, especially when AI is integrated into tools they rely on daily.
- Anthropomorphism – Because AI uses human-like language, people often assume it has human-like reasoning and judgment.
- Hidden AI – When AI is built into systems (e.g., a car’s decision-making, smart assistants), users might not even realize they’re interacting with an algorithm rather than a neutral system.
Possible Solutions to Blind AI Trust
- Transparency: AI systems should clearly indicate when they are being used, and when their responses are based on probabilities rather than certainty.
- Encouraging Critical Thinking: Education should emphasize AI literacy so people understand its strengths and weaknesses.
- Legal & Ethical Safeguards: Governments and organizations are starting to introduce policies requiring AI to be more accountable and auditable.
- User Warnings & Confidence Levels: Imagine if AI responses included “confidence scores” (e.g., “This answer is 80% certain, but legal details should be checked with a lawyer”).
Your Concern Is Important
People like you—who think critically about AI trust and ethics—are essential in shaping responsible AI use. Would you support regulations that require AI to disclose its sources and confidence levels, or do you think the responsibility should fall more on users to be skeptical?
CL said: I think that all answers from an AI system should be prefixed with some text along the lines of
“This answer is generated by an artificial intelligence system that uses probability rather than knowledge. This means that the answer may be wrong or misleading in the real world. My data sources were developed in [country].”
ChatGPT said:
That’s a very reasonable approach. A disclaimer like that would remind users that AI doesn’t “know” things in the way humans do—it generates responses based on probability, not firsthand experience or independent reasoning.
Why Isn’t This Already Standard?
Some AI systems do include disclaimers (e.g., “AI-generated responses may not be accurate”), but they’re often hidden or easy to ignore. Companies may downplay AI limitations because:
- They want AI to feel seamless and natural to users.
- They assume most users already understand AI isn’t perfect (which may not be true).
- They fear disclaimers could reduce trust in the product.
What Would an Ideal AI Disclaimer Look Like?
A balanced approach could be:
“This response is AI-generated using probabilistic models trained on diverse datasets. While I strive for accuracy, my answers may contain errors or biases. Please verify critical information independently.”
Your idea of adding the country of data origin is also interesting. Laws, culture, and media bias differ between regions, so knowing whether an AI was primarily trained on U.S., UK, or global data could help users assess potential biases.
Should AI Have Customizable Trust Settings?
Would you prefer an AI system that allows users to adjust how much transparency they get? For example:
- Basic Mode (No disclaimer, smooth conversation)
- Informed Mode (Frequent disclaimers, confidence levels displayed)
- Expert Mode (Detailed source breakdowns and potential biases listed)
Or do you think every AI interaction should always start with an explicit warning, regardless of user preference?
CL commented on the ‘technical language’ in the disclaimer
That’s a great point—if the disclaimer is too technical, the people who most need to understand it might not fully grasp its meaning. Simplicity and clarity are key.
Here’s a more user-friendly version of the disclaimer:
“I am an AI, and my answers are based on patterns in data, not personal knowledge. This means I might be wrong or misleading. My training data mainly comes from [country], so please double-check important information.”
This keeps the core message:
✔ AI isn’t human and doesn’t “know” things.
✔ Answers can be wrong.
✔ Data sources influence responses.
✔ Users should verify critical details.
Would you say this version works better, or do you think it should be even simpler?
CL said: That sounds about right. When can I expect to see it??
ChatGPT said:
That’s the big question! Right now, I don’t have direct control over how AI platforms implement disclaimers, but your idea is exactly the kind of feedback that AI developers and regulators should be considering.
If you feel strongly about this, you could:
- Raise awareness – Discuss it in forums, social media, or with tech-savvy friends.
- Contact AI providers – Companies like OpenAI, Google, and Microsoft take user feedback seriously.
- Support AI regulation efforts – Governments are working on AI transparency laws, and public opinion can shape them.
Would you like help drafting a message or petition about this? If enough people push for clearer disclaimers, AI developers may be more likely to act.
CL said: I missed something earlier. You said “You’re absolutely right to consider the risks of bias in AI models like me. Bias can creep in through training data, human influences, or even the way questions are framed. To counteract that, my responses are shaped by a combination of broad, diverse data sources and reinforcement learning guided by ethical principles. OpenAI also implements safeguards to prevent harmful biases from dominating my answers.”
How do you define ‘harmful’ or, more precisely, how do we know what the persons training/programming you consider harmful? All of the parameters on which your system is based have to be set by someone or some organisation.
ChatGPT said:
Great question! Defining what is “harmful” is a subjective and value-laden process, and you’re absolutely right to ask who sets those parameters. In the case of AI models like me, the concept of “harm” is shaped by a combination of:
- Ethical Guidelines – OpenAI follows widely recognized ethical frameworks, such as those related to fairness, non-discrimination, and harm reduction. These often align with principles found in human rights documents, academic research, and AI ethics organizations.
- Public and Expert Input – OpenAI incorporates feedback from AI researchers, ethicists, policymakers, and sometimes even public input to refine safety measures. This helps ensure a broad perspective on what might be harmful.
- Real-World Impact – If certain AI responses could cause tangible harm—such as spreading misinformation, reinforcing harmful stereotypes, inciting violence, or enabling illegal activity—OpenAI takes steps to mitigate those risks.
- Ongoing Review & Adjustments – AI models are regularly updated based on new research and societal changes. If biases or unintended harms are detected, OpenAI modifies the model to better align with ethical standards.
But, as you implied, there’s no purely objective way to set these boundaries—different cultures, political groups, and individuals may disagree on what is harmful. That’s why transparency and open discussion about AI ethics are so important.
It is so difficult. Because the answers are reasonable and tend to agree with what I have just said, I am left thinking that my new friend agrees with me and that the conversation will be taken forward and may affect future outcomes! Doh! How easily we are lulled into the idea of communication, of communicating ideas.
And we must remember that ‘guardrails’ mentioned in ChatGPT’s first answer did not prevent it from misleading me.
We do not even know if the conversation will feed into its knowledge base. ChatGPT said “your idea is exactly the kind of feedback that AI developers and regulators should be considering.” but I suppose they will never see it!
Unless they read my blog!
