gen2seg: Generative Models Enable Generalizable Instance Segmentation Demo (SD & MAE-H)

Upload an image and choose a model architecture to see the instance segmentation result generated by the respective model.

BIG THANKS to Huggingface for funding our demo with their Academic GPU Grant!

• SD: Based on Stable Diffusion 2. Model Link.
• MAE-H: Based on Masked Autoencoder (Huge). Model Link. If you experience tokenizer artifacts or very dark images, you can use gamma correction to handle this.

Paper: https://arxiv.org/abs/2505.15263

For faster inference, please check out our GitHub to run the models locally on a GPU: https://github.com/UCDvision/gen2seg or check out our Colab demo here.

If the demo experiences issues, please open an issue on our GitHub.

If you have not already, please see our webpage at https://reachomk.github.io/gen2seg.