I Tried TripoSplat to Generate 3D Gaussian Splatting from a Single Image — Then I Tried Animating a Plant
From a single image to an animated 3DGS plant: testing TripoSplat, SAM2-based organ detection, wind motion, and hydration-driven wilting.
3D generation technology is moving incredibly fast.
A few years ago, if you wanted to create a 3D model from photos, the typical workflow was something like photogrammetry: take many images, match feature points, reconstruct geometry, generate a mesh, apply textures, and clean everything up manually.
That workflow is still useful, of course. But recently, technologies like NeRF, 3D Gaussian Splatting, and single-image-to-3D generation models have made 3D content creation much more accessible.
One of the most interesting technologies in this area is 3D Gaussian Splatting, often called 3DGS.
3DGS represents a 3D scene not as a polygon mesh, but as a large number of small 3D Gaussians. Each Gaussian has properties such as position, rotation, scale, color, and opacity. When rendered from a camera view, these Gaussians are projected onto the image plane as “splats.”
In simple terms, instead of building a surface out of triangles, 3DGS recreates the appearance of a scene using many soft, semi-transparent particles placed in 3D space.
This makes 3DGS very good at reproducing realistic visual appearance. It is not always as easy to edit as a mesh, but it can look surprisingly good and render very fast. That is why it has been getting attention in games, AR/VR, simulation, robotics, and computer vision.
Recently, TripoAI released TripoSplat, which can generate 3D Gaussian Splatting data from just a single image.
Generating 3D from a single image is a very difficult task. The model cannot directly see the back side of the object, hidden parts, or the true depth. So it has to infer the missing geometry based on what it has learned from many examples.
That means the result is not the same as a real 3D scan. Hidden areas can be wrong. Shapes can be distorted. But still, the fact that we can generate a usable 3DGS file from only one image is already very exciting.
So I decided to try it.
First Test: A Coffee Bottle
For the first test, I used a simple object: an aluminum coffee bottle.
The original image had a messy background, which could confuse the 3D generation model. So before using TripoSplat, I removed the background.
For background removal, I used this Hugging Face Space:
It is very easy to use. You upload an image, and it quickly removes the background.
Then I used the cleaned image as the input to TripoSplat and generated a 3DGS file.
The output was a .ply file. To inspect it, I used the SuperSplat editor:
The result was honestly quite good.
Since the input was only one image, the bottle was not perfectly reconstructed. The original bottle is circular, but the generated shape became slightly oval. Also, the label and details on the back side, which were not visible in the input image, were not reconstructed correctly.
But the front silhouette was clean, and the color of the label was reproduced quite well. Considering that the input was only a single image, I think the result is already useful for many purposes, depending on the use case.
For example, it could be useful for quick visualization, AR/VR prototypes, simple product previews, or concept testing.
Next Challenge: A Plant
After trying a simple object, I wanted to test something much harder.
So I tried a plant.
Plants are much more difficult than bottles.
Leaves overlap each other. Stems and branches are thin. Leaves are flat and delicate. The structure is complex, and many parts are hidden behind other parts.
I used the same workflow as before:
Remove the background
Input the cleaned image into TripoSplat
Generate a 3DGS
.plyfileView it in SuperSplat
Here is the generated result.
And honestly, I thought:
This is actually pretty good.
Of course, it is not perfect. Some hidden parts are strange. Some leaves and branches are not reconstructed correctly. But again, this was generated from only one image.
For understanding the rough appearance of the plant, the result was good enough to be interesting.
At this point, I started wondering:
Can we animate this?
What If We Could Move the Plant?
Recently, there have been projects that animate 3DGS-based avatars. For example, a human character can be represented with Gaussian Splatting and then moved using a skeleton or a proxy mesh.
So I started thinking about plants.
Could we animate a plant generated as 3DGS?
For a plant, the motion I wanted to reproduce was something like this:
Leaves moving in the wind
Stems bending slightly
Leaves drooping when the plant loses water
Leaves recovering when hydration increases again
If this kind of motion could be generated from a 3DGS plant, it could be useful not only for games or AR/VR, but also for agriculture, robotics, plant simulation, and computer vision.
So I decided to build a small experimental tool for this.
I Built plant-motion-gs
The code is available here:
https://github.com/fromfactory/plant-motion-gs
The idea is simple:
Given a plant 3DGS .ply file, the program tries to detect rough plant regions such as stems, branches, and leaves. Then, in a web viewer, it animates the plant using wind and hydration parameters.
However, there is an important problem.
A normal 3DGS .ply file does not contain semantic labels like:
this Gaussian is a leaf
this Gaussian is a stem
this Gaussian is a branch
It only contains Gaussian data.
So before animating the plant, we need to estimate which Gaussians belong to which plant organs.
The pipeline roughly works like this:
3DGS PLY
↓
Render the 3DGS from multiple viewpoints
↓
Use SAM2 to estimate 2D region proposals
↓
Project the 2D masks back to Gaussian indices
↓
Estimate stems, branches, and leaves in 3D
↓
Generate data for the web viewer
↓
Animate wind and hydration
SAM2 is used to generate region proposals from the rendered 2D images. Then those masks are mapped back into the 3D Gaussian space.
This is still experimental. The current detection logic is somewhat tuned for the tomato plant structure used in this test. Other plants may not work as well, especially if they have very thin leaves, complex branching, pots, supports, or very different shapes.
But for a first experiment, it was good enough to try.
Running the Pipeline
First, place the generated plant .ply file here:
apps/viewer/public/assets/plant.ply
Then run the preprocessing pipeline:
python tools/ai_ply/run_pipeline.py \
--ply apps/viewer/public/assets/plant.ply \
--asset-dir apps/viewer/public/assets \
--work-dir work/plant_ai \
--renderer auto \
--up y \
--image-size 1024 \
--views 12 \
--elevations=-10,15,35 \
--model-id facebook/sam2.1-hiera-base-plus \
--stem-segments 18 \
--proposal-mode fine \
--rotate-deg 180,0,0
This script renders the plant from multiple viewpoints, runs SAM2-based segmentation, projects the masks back into 3D, and generates the files needed by the web viewer.
After that, install the web app dependencies and start the viewer:
pnpm install
pnpm dev
Then the plant can be viewed in the browser.
Animating Wind
The web viewer has a wind parameter.
When I increase it, the leaves and branches start moving like they are being affected by wind.
The animation is not just moving the whole plant left and right.
To make it feel more plant-like, the motion is different for stems, branches, and leaves.
The main stem moves only slightly.
The lower part stays almost fixed.
Branches inherit some motion from the stem and bend a little.
Leaves inherit branch motion, and their tips and edges move more strongly.
This is not a physically accurate simulation, but it looks much more natural than simply translating the entire 3DGS object.
Simulating Hydration and Wilting
I also added a hydration parameter.
The idea is simple:
High hydration means the plant looks healthy.
Low hydration means the plant is losing water and the leaves start to droop.
One important detail is that the main stem does not simply move downward.
When a plant wilts, the main stem usually does not just sink as one rigid object. Instead, the leaves, petioles, and branch tips tend to droop more visibly.
So in this implementation, the main stem is mostly preserved, while branches and leaf tips receive more drooping motion.
Again, this is still a simplified model. But by changing the hydration value, I was able to create a visual effect where leaves droop and then recover.
That was already pretty satisfying to see.
From One Image to an Animated 3DGS Plant
The full workflow looked like this:
One plant image
↓
Background removal
↓
TripoSplat 3DGS generation
↓
PLY inspection in SuperSplat
↓
Plant region estimation with plant-motion-gs
↓
Wind and hydration animation in a web app
There are many limitations.
The 3DGS was generated from only one image, so hidden geometry is not accurate.
The plant organ extraction is not perfect.
The animation is not a real biological or physical simulation.
The motion model is still simple.
But even with those limitations, I think the result is very interesting.
Plants are difficult objects for traditional 3D reconstruction. Leaves are thin, stems are narrow, and the overall structure is complex. Creating a clean editable mesh is not easy.
3DGS, on the other hand, is very good at capturing visual appearance. It can represent complex, fuzzy, and detailed-looking structures without requiring a clean mesh.
The challenge is that 3DGS is not naturally editable or animatable. It does not come with bones, surfaces, topology, or semantic parts.
This experiment was an attempt to bridge that gap a little.
