Building an Intelligent AI Agent for Content Moderation with Structure

Building an Intelligent AI Agent for Content Moderation with Structured Output

In this tutorial, we'll explore how to create a smart AI agent that can moderate content and generate structured reports. We'll use OpenAI's new features for structured output and function calling to build this advanced system. Our goal is to develop an AI that can analyze content effectively and provide detailed, organized results.

The Evolution of AI Outputs

OpenAI recently introduced structured output capabilities, which have changed how we work with AI models. Before this update, developers faced many challenges when using AI-generated content. The responses were usually just plain text or simple JSON formats. This caused problems for both developers and businesses.

The lack of structure in AI outputs often led to inconsistent information. Developers had to write complex code to make sense of the AI's responses, which took a lot of time and could lead to mistakes. It was also hard to control how the AI presented information, making it difficult to use these outputs in existing systems.

These limitations made it especially challenging to build reliable applications for tasks that needed precise, structured data. Teams working on legal documents, compliance reports, and business analysis found themselves spending more time dealing with AI outputs than actually benefiting from them.

The Game-Changer: Structured Outputs

But now, things have changed. OpenAI has introduced something called "structured outputs." This means the AI can now give us information in a format that's much easier to use. Imagine asking for a recipe and getting back a neatly organized list of ingredients and steps, instead of just a block of text. That's the kind of improvement we're talking about.

For our content moderation agent, this is really exciting. We can now ask the AI for specific types of information in exact formats. Need legal clauses, financial numbers, or compliance requirements? The AI can provide these in a structured way that's easy to work with. This saves a lot of time and effort in processing and organizing information.

But that's not all. OpenAI has also added something called "function calling" to its AI models. This is like giving our AI agent the ability to press buttons and pull levers based on the information it processes. It doesn't just provide data; it can take actions too.

By combining structured outputs and function calling, our agent becomes incredibly powerful. It can work with multiple sources of information, make complex decisions, and create highly customized reports. It's like having a super-smart assistant that can not only understand complex information but also do something useful with it.

This kind of AI can be really useful for businesses that need to review a lot of content quickly. It can help ensure that content meets certain standards, flag potential issues, and even suggest improvements. And because it works quickly and consistently, it can free up human moderators to focus on more complex tasks.

In this tutorial, we'll walk through how to build this kind of AI agent. We'll show you how to set it up, how to use these new features, and how to make it work for your specific needs. By the end, you'll have a powerful tool that can help with all sorts of content moderation tasks.

Let's get coding

First, create a new directory for our project:

mkdir structuredOutput

Next, let's set up a virtual environment. This will help us manage our project dependencies separately from other Python projects.

For Windows: python -m venv venv
For macOS and Linux: python3 -m venv venv

With our virtual environment activated, let's install the required libraries:

pip install openai supabase-py python-dotenv

Now, create an app.py file in the structuredOutput directory. This will be the main file for our project.

Next, create a .env file in the same directory. This file will store our sensitive information like API keys. Add the following placeholders to the file:

OPENAI_API_KEY=
SUPABASE_URL=
SUPABASE_KEY=

Don't worry if you don't have these keys yet. In the next section, we'll guide you through creating a Supabase account, setting up a table, and obtaining the necessary credentials. We'll also explain how to get your OpenAI API key if you don't already have one.

By setting up our environment this way, we're ensuring that our project is organized, our dependencies are managed, and our sensitive information is kept secure. This approach sets us up for success as we move forward with building our structured output agent.

Setting Up API Keys

Now that we have our project structure in place, let's obtain the necessary API keys for our application.

OpenAI API Key

Visit your OpenAI dashboard at https://platform.openai.com/settings/organization/general
Look for the API Keys section and create a new secret key.
Copy this key and paste it into your .env file for the OPENAI_API_KEY variable.

Introduction to Supabase

Supabase is an open-source Firebase alternative that provides a suite of tools for building scalable and secure applications. It offers a PostgreSQL database, authentication, instant APIs, and real-time subscriptions, all in one package.

We're using Supabase in this project for several reasons:

Easy setup: Supabase provides a user-friendly interface for creating and managing databases.
PostgreSQL power: It's built on top of PostgreSQL, giving us access to a robust, full-featured database.
Real-time capabilities: Supabase allows for real-time data syncing, which can be useful for collaborative document generation.
Built-in authentication: While we're not using it in this tutorial, Supabase's auth system can be valuable for securing your application in the future.
Scalability: Supabase is designed to scale with your application, making it suitable for both small projects and large-scale deployments.

Setting Up Supabase

Now, let's set up your Supabase project:

Visit https://supabase.com/dashboard/sign-up to create a Supabase account if you don't have one already.
Once logged in, click on "New Project" and follow the prompts to create a new project.
After your project is created, you'll be taken to the project dashboard.
In the left sidebar, click on your Project Homepage and scroll down to find the API section.
Here, you'll find your project URL and API key. Copy these and add them to your .env file for the SUPABASE_URL and SUPABASE_KEY variables respectively.

Your .env file should now look something like this (with your actual keys, of course):

OPENAI_API_KEY=YOUR_OPENAI_API_KEY
SUPABASE_URL=YOUR_SUPABASE_URL
SUPABASE_KEY=YOUR_SUPABASE_KEY

Next Steps

Great job! You've now set up the necessary accounts and API keys for our project. In the next section, we'll dive into creating our Supabase table, selecting the appropriate fields, and setting up the schema for our document generation system. This will lay the foundation for storing and retrieving the structured data our AI agent will work with.

Creating the Supabase Table

Now that we have our Supabase project set up, let's create the table that will store our moderation data. This table will be the backbone of our structured output system, allowing us to store and retrieve moderation results efficiently.

Steps to Create the Table

In your Supabase project dashboard, look for the sidebar and click on the "Table editor" tab.
Click on the "Create a new table" button.
Name your table MODERATION_TABLE.
Uncheck the "Enable Row Level Security (RLS)" option for now. (Note: In a production environment, you'd want to set up proper security rules.)

Setting Up the Schema

For our moderation project, we need a specific schema that can accommodate various aspects of content moderation. In the Supabase UI, you'll see a section titled "Columns" with options for "About data types" and "Import data via spreadsheet". Below that, you'll find fields for "Name", "Type", "Default Value", and "Primary".

Here's the schema we'll use:

id (text) - Set as Primary
content_id (text)
status (text)
content (text)
reported_to (text)
is_offensive (bool)
confidence_score (float4)
flagged_terms (text)
created_at (timestamp) - Set Default Value to now()
moderation_result (text)

Add each of these columns to your table using the Supabase UI. Make sure to set the correct data type for each column and mark the id column as the primary key.

After adding all the columns, click the "Save" button to create your table.

Next Steps

With our Supabase table now set up, we have a solid foundation for storing the structured output from our AI moderation agent. In the next section, we'll start building the Python code to interact with this table, including functions to insert new moderation entries and retrieve existing ones. This will form the core of our moderation system's data management capabilities.

Getting into Actual Coding

Let's break down this code into sections and then dive into each function. We'll start with the first two sections.

Imports and Initial Setup

This section sets up our environment by importing necessary libraries and initializing key components. We're using Pydantic for data validation, OpenAI for AI interactions, and Supabase for database operations. Colorama is used for colored console output, enhancing readability.

Database Operations

This function, supabase_operation, is a versatile helper for interacting with our Supabase database. It supports various operations like insert, select, update, and delete. Let's break it down:

We start by creating a query object for our MODERATION_TABLE.
Depending on the operation (insert, select, update, or delete), we modify the query accordingly.
If filters are provided, we apply them to the query. This allows for more specific database operations.
Finally, we execute the query and return the result.

This function abstracts away the complexities of database operations, making it easier to perform various actions on our moderation data throughout the rest of the code.

Data Models and Moderation Functions

Let's examine the next section of our code, which defines our data models and core moderation functions.

This section defines the core structures and functions for our moderation system:

Pydantic Models: We use Pydantic to define structured data models. ModerationResult represents the core moderation output, while ModerationOutput includes additional metadata about the moderated content.
moderate_text Function: This is our main moderation function. Here's how it works:

It takes a content_id and the content to be moderated.
It checks the content against a list of predefined offensive terms.
It calculates whether the content is offensive and assigns a confidence score.
The result is formatted into a dictionary that matches our ModerationOutput model.
The result is then inserted into our Supabase database using the supabase_operation function we defined earlier.
Finally, it returns the moderation result.

This function forms the core of our moderation system. It's a simplified version that could be expanded with more sophisticated moderation techniques, such as machine learning models or more complex rule sets.

The use of Pydantic models ensures that our data is consistently structured throughout the application, making it easier to work with and validate.

The block_content function takes a content_id as input. It's designed to mark content as blocked when it's deemed too offensive. This function creates a record in our database indicating that the content has been blocked, along with the reason. It's a crucial function for maintaining content standards on a platform.

The issue_warning function is used for content that's borderline inappropriate. It also takes a content_id and records a warning in the database. This function is useful for tracking users who frequently post questionable content or for giving users a chance to modify their behavior before more severe actions are taken.

The report_to_human function is our fallback for complex cases. It takes both the content_id and the content itself. This function flags content for review by a human moderator, which is essential for handling nuanced situations that AI might not be equipped to judge accurately.

Each of these functions returns a dictionary containing information about the action taken. They all use our supabase_operation function to insert records into the database, ensuring that all moderation actions are logged and traceable.

Initial Messages Setup

This section initializes the messages list, which is crucial for maintaining the conversation context with the AI:

The first message sets the system role, defining the AI's behavior as a content moderation assistant.
The second message simulates an initial user input to start the conversation.

These messages provide the initial context for the AI, setting the stage for the moderation process.

Main Conversation Loop

The heart of our content moderation system is the main conversation loop. This loop manages the ongoing interaction between the user and the AI, handling input, processing responses, and executing moderation actions.

Initialization

The loop begins with an initialization phase. We print a welcome message to signal that the moderation system is ready. This gives the user a clear indication that they can start interacting with the system. The use of colored text enhances the user experience.

We then enter an infinite while loop, allowing our system to continuously accept and process user input until explicitly told to stop.

User Input Handling

In each iteration, we handle user input. The system prompts the user with a green "User:" prefix. We then check if the user wants to exit the conversation by comparing their input against a list of exit commands. If they're not exiting, we append their input to our messages list, maintaining the entire conversation history.

Content Analysis

With the user's input captured, we move to the analysis phase. We indicate that content analysis is in progress and add a brief pause for better UX. The system then sends a request to the OpenAI API, passing the entire conversation history, available functions, and specifying that the AI can decide whether to call a function.

Response Processing

After receiving the API response, we process it. We check if the AI decided to call a function. If so, we extract the function name and arguments, preparing to execute the appropriate moderation action.

Function Execution

Depending on the function called, different actions are taken:

For text moderation, we call the moderate_text function and process its result. Additional actions may be taken based on the moderation output, such as blocking content or issuing warnings.
Other functions handle actions like blocking content, issuing warnings, or reporting to human moderators. Each action is executed and its result is printed for the user to see.

Regular AI Response

If no function call is made, we simply print the AI's response to the user.

Cycle Completion

This section manages exceptions and provides a final status update. The except block catches and displays any errors that occur during the main loop's execution, printing them in red for visibility. The final print statement, regardless of any errors, confirms that the moderation log has been saved to the database, displayed in green for positive reinforcement. This ensures users are informed of both errors and successful completion of the moderation process.

Running the Agent

To run the content moderation agent, follow these steps:

Open your terminal or command prompt.
Navigate to the directory containing app.py.
Run the following command:

python app.py

This will start the agent, and you'll see the initialization message. You can then begin interacting with the system by typing your messages.

Workflow of the Agent

In a real-life application, this agent would likely be part of a larger, asynchronous system. Here's how it might work:

Every message in a chat or social media platform would be sent to this agent for moderation.
The agent would process each message asynchronously, allowing for high-volume, real-time moderation.
Based on the agent's decision, messages could be:

Allowed through immediately
Blocked from being posted
Flagged for human review
Trigger warnings or actions on user accounts

The moderation results would be logged in the database for auditing and improving the system over time.

This setup allows for scalable, AI-powered content moderation that can handle large volumes of messages across multiple conversations or platforms simultaneously, providing a safer and more controlled user experience.

Conclusion

In this tutorial, we've built a sophisticated AI-powered content moderation system using OpenAI's structured output capabilities and function calling. This system demonstrates how to:

Set up a development environment for AI-driven applications
Integrate OpenAI's API for advanced language processing
Implement a database solution using Supabase for logging and data persistence
Create a robust, interactive content moderation loop

By running python app.py, you can start the agent and experience real-time content moderation. In a production environment, this system could be scaled to handle multiple conversations asynchronously, providing efficient, AI-driven moderation for various platforms.

This project serves as a foundation for building more complex AI systems, showcasing the potential of combining structured outputs with function calling in practical applications. As AI technology continues to evolve, systems like this will play a crucial role in maintaining safe and productive online environments.

Building an Intelligent AI Agent for Content Moderation with Structured Output