Building an Intelligent AI Agent for Content Moderation with Structured Output

Building an Intelligent AI Agent for Content Moderation with Structured Output

In this tutorial, we'll explore how to create a smart AI agent that can moderate content and generate structured reports. We'll use OpenAI's new features for structured output and function calling to build this advanced system. Our goal is to develop an AI that can analyze content effectively and provide detailed, organized results.

The Evolution of AI Outputs

OpenAI recently introduced structured output capabilities, which have changed how we work with AI models. Before this update, developers faced many challenges when using AI-generated content. The responses were usually just plain text or simple JSON formats. This caused problems for both developers and businesses.

The lack of structure in AI outputs often led to inconsistent information. Developers had to write complex code to make sense of the AI's responses, which took a lot of time and could lead to mistakes. It was also hard to control how the AI presented information, making it difficult to use these outputs in existing systems.

The Game-Changer: Structured Outputs

But now, things have changed. OpenAI has introduced something called "structured outputs." This means the AI can now give us information in a format that's much easier to use. Imagine asking for a recipe and getting back a neatly organized list of ingredients and steps, instead of just a block of text. That's the kind of improvement we're talking about.

For our content moderation agent, this is really exciting. We can now ask the AI for specific types of information in exact formats. Need legal clauses, financial numbers, or compliance requirements? The AI can provide these in a structured way that's easy to work with. This saves a lot of time and effort in processing and organizing information.

But that's not all. OpenAI has also added something called "function calling" to its AI models. This is like giving our AI agent the ability to press buttons and pull levers based on the information it processes. It doesn't just provide data - it can take actions too.

By combining structured outputs and function calling, our agent becomes incredibly powerful. It can work with multiple sources of information, make complex decisions, and create highly customized reports. It's like having a super-smart assistant that can not only understand complex information but also do something useful with it. This kind of AI can be really useful for businesses that need to review a lot of content quickly. It can help ensure that content meets certain standards, flag potential issues, and even suggest improvements. And because it works quickly and consistently, it can free up human moderators to focus on more complex tasks.

Let's get coding

First, create a new directory for our project:

mkdir structuredOutput

Next, let's set up a virtual environment. This will help us manage our project dependencies separately from other Python projects.

For Windows:

python -m venv env

For macOS and Linux:

python3 -m venv env

With our virtual environment activated, let's install the required libraries:

pip install openai supabase

Now, create an app.py file in the structuredOutput directory. This will be the main file for our project.

Next, create a .env file in the same directory. This file will store our sensitive information like API keys. Add the following placeholders to the file:

OPENAI_API_KEY=
SUPABASE_URL=
SUPABASE_KEY=

Don't worry if you don't have these keys yet. In the next section, we'll guide you through creating a Supabase account, setting up a table, and obtaining the necessary credentials. We'll also explain how to get your OpenAI API key if you don't already have one.

Setting Up API Keys

Now that we have our project structure in place, let's obtain the necessary API keys for our application.

OpenAI API Key

To get your OpenAI API key:

1. Visit your OpenAI dashboard at OpenAI Dashboard.
2. Look for the API Keys section and create a new secret key.
3. Copy this key and paste it into your .env file for the OPENAI_API_KEY variable.

Introduction to Supabase

Supabase is an open-source Firebase alternative that provides a suite of tools for building scalable and secure applications. It offers a PostgreSQL database, authentication, instant APIs, and real-time subscriptions, all in one package.

We're using Supabase in this project for several reasons:

• Easy setup: Supabase provides a user-friendly interface for creating and managing databases.
• PostgreSQL power: It's built on top of PostgreSQL, giving us access to a robust, full-featured database.
• Real-time capabilities: Supabase allows for real-time data syncing, which can be useful for collaborative document generation.
• Built-in authentication: While we're not using it in this tutorial, Supabase's auth system can be valuable for securing your application in the future.
• Scalability: Supabase is designed to scale with your application, making it suitable for both small projects and large-scale deployments.

Setting Up Supabase

Now, let's set up your Supabase project:

1. Visit Supabase Sign Up to create a Supabase account if you don't have one already.
2. Once logged in, click on "New Project" and follow the prompts to create a new project.
3. After your project is created, you'll be taken to the project dashboard.
4. In the left sidebar, click on your Project Homepage and scroll down to find the api section.
5. Here, you'll find your project URL and API key. Copy these and add them to your .env file for the SUPABASE_URL and SUPABASE_KEY variables respectively.

Your .env file should now look something like this (with your actual keys, of course):

OPENAI_API_KEY=
SUPABASE_URL=
SUPABASE_KEY=

Next Steps

Great job! You've now set up the necessary accounts and API keys for our project. In the next section, we'll dive into creating our Supabase table, selecting the appropriate fields, and setting up the schema for our document generation system. This will lay the foundation for storing and retrieving the structured data our AI agent will work with.

Creating the Supabase Table

Now that we have our Supabase project set up, let's create the table that will store our moderation data. This table will be the backbone of our structured output system, allowing us to store and retrieve moderation results efficiently.

Steps to Create the Table

1. In your Supabase project dashboard, look for the sidebar and click on the "Table editor" tab.
2. Click on "Create a new table" button.
3. Name your table MODERATION_TABLE.
4. Uncheck the "Enable Row Level Security (RLS)" option for now. (Note: In a production environment, you'd want to set up proper security rules.)

Setting Up the Schema

For our moderation project, we need a specific schema that can accommodate various aspects of content moderation. In the Supabase UI, you'll see a section titled "Columns" with options for "About data types" and "Import data via spreadsheet". Below that, you'll find fields for "Name", "Type", "Default Value", and "Primary".

Here's the schema we'll use:

• id (text) - Set as Primary
• content_id (text)
• status (text)
• content (text)
• reported_to (text)
• is_offensive (bool)
• confidence_score (float4)
• flagged_terms (text)
• created_at (timestamp) - Set Default Value to now()
• moderation_result (text)

Add each of these columns to your table using the Supabase UI. Make sure to set the correct data type for each column and mark the 'id' column as the primary key.

After adding all the columns, click the "Save" button to create your table.

Next Steps

With our Supabase table now set up, we have a solid foundation for storing the structured output from our AI moderation agent. In the next section, we'll start building the Python code to interact with this table, including functions to insert new moderation entries and retrieve existing ones. This will form the core of our moderation system's data management capabilities.

Getting into Actual Coding

Let's break down this code into sections and then dive into each function. We'll start with the first two sections.

Imports and Initial Setup

This section sets up our environment by importing necessary libraries and initializing key components. We're using Pydantic for data validation, OpenAI for AI interactions, and Supabase for database operations. Colorama is used for colored console output, enhancing readability.

Database Operations

This function, supabase_operation, is a versatile helper for interacting with our Supabase database. It supports various operations like insert, select, update, and delete. Let's break it down:

• We start by creating a query object for our MODERATION_TABLE.
• Depending on the operation (insert, select, update, or delete), we modify the query accordingly.
• If filters are provided, we apply them to the query. This allows for more specific database operations.
• Finally, we execute the query and return the result.

This function abstracts away the complexities of database operations, making it easier to perform various actions on our moderation data throughout the rest of the code.

Data Models and Moderation Functions

Let's examine the next section of our code, which defines our data models and core moderation functions.

This section defines the core structures and functions for our moderation system:

• Pydantic Models: We use Pydantic to define structured data models. ModerationResult represents the core moderation output, while ModerationOutput includes additional metadata about the moderated content.
• moderate_text Function: This is our main moderation function. Here's how it works:
• It takes a content_id and the content to be moderated.
• It checks the content against a list of predefined offensive terms.
• It calculates whether the content is offensive and assigns a confidence score.
• The result is formatted into a dictionary that matches our ModerationOutput model.
• The result is then inserted into our Supabase database using the supabase_operation function we defined earlier.
• Finally, it returns the moderation result.
• This function forms the core of our moderation system. It's a simplified version that could be expanded with more sophisticated moderation techniques, such as machine learning models or more complex rule sets.
• The use of Pydantic models ensures that our data is consistently structured throughout the application, making it easier to work with and validate.

Blocking and Reporting Functions

The block_content function takes a content_id as input. It's designed to mark content as blocked when it's deemed too offensive. This function creates a record in our database indicating that the content has been blocked, along with the reason. It's a crucial function for maintaining content standards on a platform.

The issue_warning function is used for content that's borderline inappropriate. It also takes a content_id and records a warning in the database. This function is useful for tracking users who frequently post questionable content or for giving users a chance to modify their behavior before more severe actions are taken.

The report_to_human function is our fallback for complex cases. It takes both the content_id and the content itself. This function flags content for review by a human moderator, which is essential for handling nuanced situations that AI might not be equipped to judge accurately.

Each of these functions returns a dictionary containing information about the action taken. They all use our supabase_operation function to insert records into the database, ensuring that all moderation actions are logged and traceable.

These functions work together to create a comprehensive moderation system, allowing flexibility and nuance in content moderation.

Initialization and Schema Definition

Now we need to initialise the client and define our schemas

First, we initialize the OpenAI client. This client is our gateway to interact with OpenAI's powerful language models. The API key is crucial as it authenticates our requests to OpenAI's services.

Function Schema Definitions

Now, let's look at each function schema individually:

• The moderate_text_function schema defines how our AI should understand and use our main moderation function.
• The block_content_function schema is used when content needs to be blocked outright.
• The issue_warning_function schema describes how to issue a warning for borderline content.
• The report_to_human_function schema is used when the AI determines that human intervention is necessary.

We compile all these function schemas into a single list, representing the complete toolkit available to our AI for content moderation.

Initial Messages Setup

This section initializes the messages list, which is crucial for maintaining the conversation context with the AI:

• The first message sets the system role, defining the AI's behavior as a content moderation assistant.
• The second message simulates an initial user input to start the conversation.

Main Conversation Loop

The heart of our content moderation system is the main conversation loop. This loop manages the ongoing interaction between the user and the AI, handling input, processing responses, and executing moderation actions.

1. Initialization

The loop begins with an initialization phase. We print a welcome message to signal that the moderation system is ready, giving the user a clear indication that they can start interacting.

2. User Input Handling

In each iteration, we handle user input. The system prompts the user with a green "User:" prefix and checks if the user wants to exit the conversation by comparing their input against a list of exit commands.

3. Content Analysis

With the user's input captured, we send a request to the OpenAI API, passing the entire conversation history, available functions, and specifying that the AI can decide whether to call a function.

4. Response Processing

After receiving the API response, we process it. We check if the AI decided to call a function. If so, we extract the function name and arguments, preparing to execute the appropriate moderation action.

5. Function Execution

Depending on the function called, different actions are taken, such as blocking content or issuing warnings. Each action is executed, and its result is printed for the user to see.

6. Cycle Completion and Error Handling

This section manages exceptions and provides a final status update. We confirm that the moderation log has been saved to the database, ensuring users are informed of both errors and successful completion of the moderation process.

Running the Agent

To run the content moderation agent, follow these steps:

1. Open your terminal or command prompt.
2. Navigate to the directory containing app.py.
3. Run the following command: python app.py.

This will start the agent, and you'll see the initialization message. You can then begin interacting with the system by typing your messages.

Workflow of the Agent

In a real-life application, this agent would be part of a larger, asynchronous system:

• Every message in a chat or social media platform would be sent to this agent for moderation.
• The agent would process each message asynchronously, allowing for high-volume, real-time moderation.

Based on the agent's decision, messages could be allowed through immediately, blocked from being posted, flagged for human review, or trigger warnings on user accounts. The moderation results would be logged in the database for auditing and improving the system over time.

Conclusion

In this tutorial, we've built a sophisticated AI-powered content moderation system using OpenAI's structured output capabilities and function calling. Key takeaways include:

• Setting up a development environment for AI-driven applications.
• Integrating OpenAI's API for advanced language processing.
• Implementing a database solution using Supabase for logging and data persistence.
• Creating a robust, interactive content moderation loop.

This project serves as a foundation for building more complex AI systems and showcases the potential of combining structured outputs with function calling in practical applications. As AI technology continues to evolve, these systems will play a crucial role in maintaining safe and productive online environments.