GIMPIX: Video AI Software Analysis |
We have tested (or are fixing to) some of the currently available video AI software to ascertain suitability for our client desires.
Static image from text software has already reached the point of being able to produce highly realistic retro cast images a decent percentage of the time. Does it still generate rubbish? Yes, frequently, but will likely only improve. It's certainly a quantum leap over crude image editing techniques we used yesteryear. Video will add a whole new dimension and at Gimpix we don't think it's hyperbole to say that coupled with VR, video AI is the closest we will likely come to inventing a time machine. So where do we stand as of late 2024? Sora from Open AI is still in the beta testing stage in house despite a promised 2024 release date. There are some other video AI software products available though so where do they stack up?
images used for testing
The prompt(s) we are using for these tests will be:
full view of 1960s woman walking in supermarket (simple 'control' prompt to test basic concepts)
and
1960s woman with leg in plaster cast from toes to upper thigh walking on wooden crutches in supermarket (more complex)
|
Product |
Control |
Sample |
Comments |
Aitubo.ai |
Ctrl 01 |
Ctrl 01 |
There does not appear to be a free trial of the video generator. Using the image generator it appears that this product also does not grasp the key concepts we're looking for. |
|
Haiper |
Ctrl 01 |
|
Has options for both text to video and image to video. Cast video results were among the worst we've seen so far. We don't think it's even worth the space on the server to upload the cast sample! The control sample is not horrible but lacks the smoothness of other products. We actually tested this product with text to video back in March 2024. It does not appear to have improved by using an image. |
|
Kling |
Ctrl 01 |
Cast 01 |
The first challenge is signing up! On the human verification screen it is far from obvious that the way to drag the puzzle piece is via the arrows below! Once into the interface one gets 365 or so free credits to use although the pricing plans seem quite reasonable. Our first sample took a good 30 minutes to generate and it wants both a source image & prompt. The first control sample looks very promising! The cast sample however, was a disappointment. The software does not appear to grasp the concept of crutching motion or the cast remaining rigid. We went ahead and sent that feedback. However, when we substituted the word 'hobbling' for 'walking', the results became much better: Cast 02 This might also have been a factor with Minimax? Kling has a variety of monthly plans starting at under $9/mth (660 credits) to $81/mth (8000 credits). Our samples used around 35 credits for reference. |
|
Krea.ai |
|
|
This appears to be the 'Trivago' of: Hailuo, Luma, Runway and Kling (std/pro). We tried to give it our complex prompt using the Hailuo model but there does appear to be a free trial. |
|
LTX Studio |
|
|
Appears to only do storylines? |
|
Luma Labs Dream Machine |
Ctrl 01 |
Cast 01 |
The control sample gets the basics right although the motion is jerky but once the complexity of a cast & crutches are introduced the software does not know how to interpret those items. This one has a long way to go to be useful. |
|
Mid Journey |
|
|
.. |
|
Minimax aka Hailuo |
Ctrl 01 |
Cast 01 |
Site is hailuoai.video Signup is simple and you get a free trial. We gave it the same sample photo & prompt as Kling. Video generation was quite a bit quicker. This one looks promising. The standard plan at $9.99/mth gives 1000 credits or they can be purchased in various packages. $10=1000 so it's a bit of a wash but might make sense for continued evaluation purposes. Now, if you've resisted clicking on the cast sample, OMG!, this is both exciting and a bit bloody scary! Notice how her expression changes from smiling to grimacing as she struggles in her big cast! The crutching motion is not quite there but that might also be a function of video length. We'll see if the paid version allows longer video. This one looks very promising. They might've beaten Sora to the punch. WINNER thus far? What about if we try animating a real cast image? Cast 02 Holy s**t. Not sure what she's doing right at the end but this has so much potential. It does not quite get the concept of descending stairs though: Cast 03 |
|
Pika |
|
|
We also tried this one some months back using a woman skiing (from a text prompt) Whilst it got the basics right, the motion quality left a great deal to be desired. We were unable to generate a new sample as the server was too busy. |
|
Pixverse |
|
Cast 01 |
We blew 30 of our daily 60 free credits on a bad setting so only have this sample so far using 'walking'. Credits renew daily so we'll try again later using 'hobbling' along with the control sample. At $8, the standard plan gives 30 daily renew credits and 1200 monthly so this one shows some promise. |
|
Pollo |
|
Cast 01 |
We used a slightly enhanced version for this one: '1960s woman with leg in plaster cast from toes to upper thigh hobbling on wooden crutches in supermarket keeping cast rigid and non bending'. The results in 01 are quite promising despite some cast flex and the 'floating' along that Kling & Hailuo (Minimax) sometimes succumb to. We also tried the first sample with only 50% prompt strength (read adherence). Upping it to 75% actually produced worse results. One gets the impression that the interface is very similar to Kling and Hailuo so is this truly a different product? The pricing structure would suggest so. The free version uses only 10 credits for standard videos as does the lite version for $10 month which provides 300 credits. There does not seem to be any (significant) bonus for daily login. The pro version if this emerges as a winner is quite flexible allowing one to tailor credits desired to monthly outlay up to 50,000 credits for $659! |
|
PromeAI |
Ctrl 01 |
Cast 01 |
Site is promeai.pro It also uses a combination of image and prompt. Whilst it did OK on the control image albeit with nowhere near the smoothness of Kling & Minimax, it totally fell down when the cast concept was introduced |
|
Runway (Gen.3) |
Ctrl 01 |
Cast 01 |
Re-directed to imagine.art Both the control and the cast samples are disappointing. This product has a long way to go still. |
|
Stable video diffusion |
Ctrl 01 |
Cast 01 |
Site: stablevideo.com v1.1 Anything but stable in our view. The faces and other features are horribly distorted and the concept of crutching non-existent. Pass on this one for now. |
|
Sora |
|
|
This is eagerly awaited. It was promised sometime in 2024. Better get cracking Sora! Looks like Kling and Minimax have upstaged you! |
|
videoexpress.ai |
|
|
This is old school software in that it appears to be a one time price for the license. Sadly, there did not appear to be any kind of trial period to enable us to test this made in Canada product which looks like it might be able to easily add voice to video? We'll email them 'aboot' that eh? |
|
Videogen.io |
|
|
Has zero concept of elements we're looking for. |
|
Vidu.ai |
|
|
Appeared to re-direct to: aihugging.org No free trial so not able to generate samples and the starter plan at $7.99 month doesn't even allow downloads. Not much use we reckon. |
|
Static Image Generators: Since many of the video generators enable the use of an image as a starting point a review of the current offerings is probably in order. |
Dall E-3 |
|
|
This is the one we have been using via Bing. It certainly understands the concept of a cast and generated some decent images. However, the ones we have published are the better ones (although there are other good ones awaiting upload). Many, many have been rejected as outright rubbish. We have recently been able to convey to the it the concept of writing on a cast although sometimes it ends up all over the walls as well! Some concepts that it really struggles with are:
1. Eliminating the non casted leg in sitting poses. We have tried all sorts of different language to no consistent avail. It does what it wants and if you get both legs its a fluke a small percentage of the time.
2. Cast elevated in traction: Again, we've tried to convey this is many ways without much success.
3. Walking cast with (rubber) heel. Must admit we've not really tried the older style walking iron.
4. Getting basics wrong (sometimes) like the toes going the wrong way for the leg side. Add to that too many or (less often) too few toes.
All things considered this is a decent product and it's free. Even when one uses up the daily 15 'boosts', it's still possible to keep going for quite a long time. We've not really noticed any difference in image accuracy with or without the boosts, in fact, some time after boosts run out is when it can produce a near gem. Learning a new concept?
Generally, Dall E3 does best with standing poses and almost never gets it wrong on these as described in number one above.
|
|
Mid Journey |
|
|
Used openart.ai with both the prompt at the top of the page (standing instead of walking) and one of our more detailed prompts. Dismal results for both. |
|
|
|
|
.. |
|
|
Copyright (c) Gimpix. All rights reserved
|