Hackaday · Maken

Running Generative AI on an RP2350

A project demonstrates running an image generation diffusion model on an RP2350 microcontroller, driven by interests in privacy, customization, and cost reduction for local AI.

The model generates 128x128 images of human faces, with each image taking approximately twenty seconds to produce, which is notable given the hardware's limitations. The system utilizes a Waveshare RP2350 development board and can output images via USB or display them using a VGA adapter.

The image generation process involves creating a distribution in a latent space, which is then translated into an image by the decoder component of a variational auto-encoder. The auto-encoder was trained in two stages: an encoder to map images to latent-space distributions and a decoder to reconstruct images from these distributions. For generation, only the decoder is used.

The generative part of the model employs a latent flow diffusion transformer. It begins with noise and iteratively refines it to approach the target image. The model can also be guided by an output class, such as generating a smiling face.

[Tim] trained two models, one optimized for speed and another for size, and quantized their weights to 8-bit integers. Both models and the inference program fit within 4 MB of flash memory. The generated images, while not perfectly natural, are recognizable.

AI-samenvatting op basis van de bron.