Building an App with Aria and Allegro: Create Fun Fact Videos from Travel Photos

Building an App with Aria and Allegro: Turning Travel Photos into Fun Fact Videos

Hello! It’s Tommy here, and today, I’m excited to walk you through a project where we’ll transform travel photos into fun fact videos. Using Rhymes AI’s Aria API to analyze images, we’ll generate rich scene descriptions and bring them to life with Allegro’s text-to-video model. This tutorial lets you explore the creative potential of these tools in a fun, hands-on way.

Whether you’re looking to experiment with multimodal APIs or curious about unique app integrations, this guide will help you adapt these tools to suit your projects. Stick around until the end for a link to the Colab notebook so you can follow along directly.

Getting Started with the Setup

To begin, let’s set up our environment and install the necessary libraries. Here’s what you’ll need:

• Python 3.x
• Required Libraries: Rhymes AI, Requests, and any other dependencies.

Once we’ve installed the requirements, we can move to the image preparation and API integration sections.

Preparing Your Image in Base64 Format

The first step is to convert your image into base64 format, which will allow us to send it through the Aria API. Here’s a function to handle the conversion:

def image_to_base64(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode('utf-8')

Usage: Provide your image path to image_to_base64() to get the base64-encoded string.

Analyzing the Image with Aria’s API

Now that we’ve prepared the image, let’s use Aria’s multimodal API to analyze it. This API will return a set of scene descriptions that bring the location in the photo to life. Be sure to replace userdata.get('ARIA_API_KEY') with your own API key, or update the secret in Colab with the same parameter.

Creating a Video Task with Allegro

Let’s now use Allegro’s text-to-video API to create a video based on the scene descriptions. This function initiates a video generation task, which we’ll query in the next section using the request_id returned here.

Remember to replace userdata.get('ALLEGRO_API_KEY') with your actual Allegro API key or set it as a Colab secret with the same parameter.

Usage: Replace userdata.get('ALLEGRO_API_KEY') with your Allegro API token. Run the function and capture the request_id, which we will use to query the video status.

Note: When calling the create video task endpoint, be aware that if you hit the endpoint again within a 2-minute interval, you may encounter an error message: "The request rate for model Allegro has exceeded the allowed limit. Please wait and try again later". This response comes with a status code of 500, indicating that a brief wait between requests is required to avoid rate limiting.

Checking the Video Generation Status

Because Allegro can take around 2 minutes to process the video, we’ll add a time.sleep() delay before querying.

When you run this, Allegro will return a link to the video stored in an S3 bucket:

Displaying the Generated Video Image

Here’s how the generated video might look:

Once the video link is retrieved, I captured a screenshot from the video to showcase the result. This visual gives you an idea of what the final output could look like when you follow these steps to transform a travel photo into a dynamic video.

Find the link to the Google Colab Notebook for this tutorial here.

Wrapping Up

Congratulations! You’ve successfully created an app that transforms a travel photo into a fun fact video. By using Aria to generate compelling scene descriptions and Allegro to bring them to life in video format, you’ve tapped into the potential of multimodal AI applications.

For further customization and a more advanced setup, check out the detailed documentation here. This tutorial opens the door to endless possibilities with Aria and Allegro, whether you’re crafting travel-inspired content, educational materials, or any other creative media.

Enjoy exploring, and let your imagination guide you to new ideas and projects!

Next Steps

Here are some practical steps to expand your app:

• Integrate more APIs for enhanced functionality.
• Add user authentication to personalize content.
• Experiment with different video formats and styles.