E Tutorial: How to Set Up and Use Shape-E for 3D Model Generation

What is Shape-E?

Shap-E is an innovative model developed by OpenAI that revolutionizes the generation of 3D objects using text or image inputs. This transformative technology opens new doors in the realm of 3D applications by allowing users to create unique models easily. What's even more appealing is that Shap-E is freely accessible on GitHub, meaning users can run it on their own computers without needing an OpenAI API key or internet connectivity. Its versatility is one of its standout features, as the 3D objects produced can be opened in various design software, including Microsoft Paint 3D, and can be transformed into STL files suitable for 3D printing. With Shap-E, the process of text-to-3D and image-to-3D generation is not only possible but remarkably user-friendly, encouraging creative exploration with AI applications.

What Will We Do?

In this comprehensive tutorial, we will guide you through creating a Notebook in Google Colab, setting up and utilizing OpenAI's Shape-E model to generate 3D models, and customizing those models using Blender Studio.

Prerequisites

Before we begin, ensure you have the right tools. Visit Blender.org to download Blender Studio compatible with your operating system.

Let's Start!

Step 1 - Setting Up the Project

Begin by creating a new Notebook in Google Colab:

1. Go to Google Colab and create a new Notebook.

Step 2 - Clone the Repository

In order to use Shap-E, we need to clone its repository into our Google Colab Notebook:

1. Enter the project directory in the Notebook.
2. Install the necessary requirements listed in the repository.

Step 3 - Implementing and Generating 3D Models

Now, let's begin coding:

1. Add a new code cell to import the required libraries.

Click the Run button or use CMD/CTRL + Enter to execute the cell.

Next:

1. Add another new code cell to set the device to CUDA if available; otherwise, default to CPU.
2. Click Run or CMD/CTRL + Enter.
3. Add another code cell to load the models and weights. Click Run or CMD/CTRL + Enter.

Be patient; loading the models and weights will take some time, depending on your internet speed.

Generating 3D Models

Adding one more code cell now will allow us to generate the 3D models:

1. Click Run or CMD/CTRL + Enter.

The generation might take a bit longer based on the batch size; for example, it took me around 22 seconds to generate models with a batch_size=1.

Rendering the 3D Models

We can further enhance our work by rendering the 3D models:

1. Add a new code cell and set render_mode = 'nerf' (Neural Radiance Fields) to visualize the models.
2. Alternatively, change it to render_mode = 'stf' for Style Transfer Function rendering.
3. Click Run or CMD/CTRL + Enter.

Did you enjoy watching the models rotate? It’s quite impressive!

Saving the 3D Models

Now we can save the models in .ply and .obj formats:

1. Add a new code cell and execute it.
2. Go to the Files tab and refresh it to see your saved files (example_mesh_0.ply and example_mesh_0.obj).
3. Download the .obj file to your local machine.

Step 4 - Customizing the 3D Models in Blender Studio

Open Blender Studio and create a new project:

1. Delete the default cube.
2. Go to File > Import > Wavefront (.obj) and select the downloaded .obj file.
3. You should see your 3D model appear in the center of the screen.

Isn’t it fantastic? Shap-E has done a wonderful job!

You can now customize the model as desired. For instance, I added pink to the topping and brown to the donut.

Conclusion

In this tutorial, we have successfully set up and utilized the Shape-E model via Google Colab and explored Blender Studio for customizing generated 3D models. Now it's your turn to experiment with the skills you've acquired and create your own 3D art. As you dive deeper, you might encounter limitations and consider developing your own Shap-e application.

Participate in AI hackathons, showcase your creativity, and win exciting prizes! Join the community of AI enthusiasts to delve deeper into AI's capabilities and future potential!

Thank you for reading this tutorial. Happy creating!