Ethical AI

“AI technology holds immense potential to address societal challenges at scale, but it won’t do it on its own—it has to be actively shaped by actors whose mandate is to put the public interest first. Mission-driven organizations are essential for creating technology that truly serves humanity’s needs.”

(https://www.quill.org/ethical_ai)

Quill is a Brooklyn-based nonprofit created by “Peter Gault and some Brooklyn-based dreamers who, inspired by the possibility of the internet to give everyone a voice, imagined a world in which all people had the writing and critical thinking skills to engage in active, intelligent debate and discourse.” Their “Ethical AI Hub” features a “Generative AI Playbook” that emphasizes tailoring AI frames to human behavior challenges, such as learning and critical thinking. Quill says it has developed more than 300 unique AI models,  a collection of specialized, smaller AI programs developed to evaluate and provide feedback on specific aspects of student writing. Quill.org focuses on “AI that writes with you, not for you.”

According to Gemini (10/28/2025):

Quill builds what it calls a “thick wrapper” around its AI, which provides the model with extensive, human-written instructions and examples. This approach ensures the AI’s behavior is guided by pedagogical intent rather than an unguided interpretation of a training dataset.
  • Controlled environment: For text-dependent prompts, students must base their writing on the same source material. This provides the AI with a “source of truth” to reference, making it more difficult for the model to inaccurately assess content based on factors like a student’s prior knowledge or personal background.
  • Specific and targeted feedback: Feedback is not generic but instead tied to the source text and the specific task. This prevents feedback from inadvertently favoring certain writing styles that might be associated with particular demographic or socioeconomic backgrounds.
Quill has a continuous process of testing and evaluation to monitor the effectiveness and fairness of its AI models.
  • Continuous human evaluation: Full-time curriculum developers manually evaluate over 100,000 student responses each year to ensure the feedback quality is high and free of bias.
  • Benchmark datasets: The company creates benchmark evaluation datasets—distinct from its training data—to rigorously assess the AI’s performance and reliability.
  • A/B testing: Quill uses A/B testing to compare different approaches and identify where improvements are needed to make the AI more effective and equitable.
Focus on conventions, not content
Quill’s AI is designed to focus on conventions of grammar, syntax, and evidence-based reasoning, rather than subjective stylistic judgments or content. The feedback often asks students to improve clarity or precision within the context of the provided text, avoiding comments that might unintentionally devalue different cultural linguistic practices. For instance, rather than simply labeling something as “incorrect,” the feedback might guide a student toward a more precise or conventionally accepted phrasing, while also acknowledging the legitimacy of their original language.
This approach has several interesting features. First, that it is capable of evolving its learning models over time, whereas the mainstream LLMs like Grok and ChatGPT do not actually learn from their interactions with humans. This seems like an obvious next step. According to Gemini:
Next-generation AI systems are being designed to learn from their interactions with humans, a concept known as socially situated or human-assisted AI. This involves AI learning from ongoing conversations and feedback in real-world social environments to develop new concepts, understand social norms, and improve their capabilities.
Interestingly, “Studies show that AI systems can develop neural patterns similar to those in a biological brain when they engage in social interactions, a convergence that can help create more socially aware AI.” This leads to Augmented Intelligence. “The goal is often for AI to collaborate with humans, augmenting human capabilities rather than replacing them. This human-AI interaction is seen as a way to enhance decision-making and creativity, as discussed by MIT Sloan and The Interaction Design Foundation.
Here’s what Gemini says on a “deeper dive”:
Yes, next-generation AI systems will learn from their interactions with humans, and many already do. This dynamic is driven by advanced machine learning techniques that allow AI models to continuously adapt and improve based on real-world feedback, a process known as continual learning.
This learning from human interaction extends beyond a model’s initial training phase, influencing its ongoing behavior and personalization.
How AI learns from human interaction
1. Reinforcement Learning from Human Feedback (RLHF)
This technique is a cornerstone of modern generative AI models, including large language models (LLMs). It works by using human judgments to train a “reward model,” which teaches the AI to produce more helpful and aligned responses.
  • The human feedback is often in the form of ranking multiple AI-generated responses from best to worst.
  • The reward model, in turn, helps fine-tune the LLM to prioritize the preferences expressed by humans.
2. Continual learning
Inspired by how humans learn incrementally, continual learning enables AI models to integrate new information over time without “catastrophic forgetting,” a phenomenon where a model loses old knowledge when learning something new.
  • This approach is essential for keeping AI systems up-to-date in dynamic, real-world environments.
  • For example, a continual learning AI for a self-driving car can learn to identify new types of objects on the road without forgetting how to recognize pedestrians, signs, or other vehicles.
3. Socially situated AI
This learning framework, which is inspired by how children learn through interaction, enables AI agents to learn new concepts by engaging with people in real-world social environments, such as a social media platform.
  • Researchers have developed AI agents that can ask humans natural language questions about images to learn new visual concepts and adapt their interaction style to keep humans engaged.
  • This approach moves away from training on static, fixed datasets and toward active learning through ongoing, human-centered interaction.
The future of human-AI learning dynamics
The integration of learning from human interactions is central to many predicted future AI trends:
  • Multimodal AI: These next-gen systems will process information across multiple data types, such as visuals, text, and voice, to create more natural and intuitive human-computer interactions.
  • Personalized adaptation: As AI becomes more integrated into our lives, it will use human interaction to learn user preferences and behaviors. This will enable more personalized experiences in everything from virtual assistants to educational platforms and customer service.
  • Agentic AI: Future AI agents will be able to anticipate human needs and make autonomous decisions by learning from their environments and past interactions. They will be composed of specialized agents that handle specific tasks, making them more adaptable and efficient.

This aligns with much of the thinking here, that AI needs to evolve alongside the humans interacting with it.