A Complete Guide to Setup and Use Shap-E for 3D Modeling

What is Shape-E?

Shape-E, a groundbreaking model developed by OpenAI, produces an array of 3D objects using text or images as input. This innovative model is transforming the realm of 3D applications with its unique approach. Best of all, Shape-E is accessible for free on GitHub, allowing users to run it seamlessly on their computers without requiring an OpenAI API key or an internet connection.

What truly sets Shape-E apart is its versatility: the generated 3D objects can be opened in software like Microsoft Paint 3D or converted into STL files for 3D printing. This model is redefining the possibilities of text-to-3D and image-to-3D generation, opening up a world of creative opportunities for AI applications.

What Will We Do?

In this tutorial, we will learn how to create a Notebook in Google Colab, set up OpenAI's Shape-E model to generate 3D models, and customize them using Blender Studio.

Prerequisites

• Download Blender Studio from Blender.org compatible with your operating system.

Let's Start!

Step 1 - Setting up the Project

Begin by creating a new Notebook in Google Colab.

Step 2 - Clone the Repository

• Clone the Shap-E repository to your Google Colab Notebook.
• Enter the directory and install the requirements.

Step 3 - Implementation and Cooking the 3D Models

Add a new code cell and import all the necessary libraries.

Run the code cell (CMD/CTRL + Enter) to execute.

Add another code cell to set the device to cuda if available; otherwise, it should be set to cpu. Execute the code cell.

Next, load the models and weights in a new code cell. This process may take some time, approximately 5 minutes depending on your internet connection.

• Add a code cell to generate the 3D models, which will take time based on your batch_size. For example, generating a 3D model with batch_size=1 took about 22 seconds.
• Add a code cell for rendering the 3D models using render_mode = 'nerf' (Neural Radiance Fields). If you prefer, change this to render_mode = 'stf' for Style Transfer Function rendering.

Once the models are generated, add a new code cell to save the 3D models as .ply and .obj files. Note, you will later use the .obj file for customization in Blender Studio. Refresh the Files tab in Colab to download example_mesh_0.obj.

Step 4 - Customizing the 3D Models in Blender Studio

Open Blender Studio and create a new project.

• Delete the default cube.
• Import the .obj file by navigating to File > Import > Wavefront (.obj) and selecting the downloaded file.

The 3D model should now appear at the center of your workspace! You can now customize it as you would like. For demonstration purposes, you might color the topping pink and the base donut brown.

Conclusion

In this tutorial, we explored how to set up and use the Shape-E model in Google Colab and customize a generated 3D model using Blender Studio. Now it’s your turn! Test out the skills you’ve learned and start creating your own 3D art.

As you experiment, consider the potential limitations. Perhaps you want to enhance Shape-E with your own application? Why not participate in an Artificial Intelligence Hackathon and showcase your creativity while competing for great prizes! Join the community of AI enthusiasts and learn more about what AI can achieve.

Thank you for reading this tutorial!