This one took the long way 'round to get to the reveal. Image gen is almost always surprising, and frequently makes me laugh.
Starting full size Flux image:
Ok, bear with me here. I’ll say upfront, not interested in the nudity, but I might have some use case for video generation - although what I’m really interested in is the workflow.
You mention flux, so I assume you’re talking about the FLUX.1 model and you used that to generate the initial image from a prompt. Your profile also says everything is solar powered, so does that imply you’re running all models on your own infrastructure?
In any case, what did you use to generate the video then? And if that’s more or less self-hosted, what are today’s requirements for running video generation, both in hardware and time?
Yes, this is all local and self-hosted.
On the hardware front, I’m running an Intel 12900k, Nvidia 3090, 64GB ram. The workflow is all done with ComfyUI. Starting with a single image from the Flux 1 Dev model (text to image). Then passing that to a new video model called WAN 2.1 for the image-to-video.
The Flux image takes about 2 minutes to generate but I make them big, about 2.5 megapixels with an upscale/finetune pass. The video takes about 3-4 hours to output 10 seconds of 720p at 15fps. Any larger res or more frames and I get out of memory errors. After that, I run it through a script I wrote using a frame interpolator called RIFE to get it up to 30fps.
If you don’t need NSFW and don’t mind a Chinese company doing the video, I’ve gotten great image-to-video results from KlingAI in just a few minutes. The company behind Flux is also working on a local i2v model, and there’s another new one called HunyuanVideo that just came out a few days ago but I haven’t tried it.
Very interesting info - there is some really useful stuff in there I can use as an immediate starting point, so thank you for that!
