Working with Videos on 3D Objects in Resolume
I had the opportunity to work closely with the Scandy team in developing an AR and mixed media installation to be shown during South by Southwest in Austin. Unfortunately, due to the COVID-19 pandemic, the conference was canceled and we weren’t able to showcase the project. Since we weren’t able to do the live event, I put together some short clips I made in Resolume using 3D objects.
Scandy has a ton of features that make it really easy to get great 3D videos. It’s in-app recording function is perfect for capturing short movements to loop in Resolume. You can also play around with the background colors or wireframe view to get some variation in your content. Ultimately I found it easier to control the objects’ movements with a mouse rather than a touchscreen, so I used QuickTime to screen record in Meshmixer and Blender on my computer. Whichever route you take might depend on the type of movement you are looking for. Something to note: make sure that you avoid recording your mouse or any of the graphic interfaces while recording in those 3D applications, unless you plan on post processing the video. (The beauty of the Scandy record function is that you don’t have to worry about any of that.)
Once I put all my clips into Resolume, I found it really useful to add a Posterize effect. Since most of my videos were recorded with grey backgrounds, adding the Posterize effect pushed the grey to black. This makes it easier to add layers and allow them to show all the way through. Otherwise, lowering the opacity and testing out some blending modes on the layer could also do the trick.
I was browsing through a collection of Scandy Pro scans on Sketchfab when I came across these three scanned faces that were blended together. I took a QuickTime video of the faces turning 360 degrees and went from there. In Resolume I added the recolour effect to the masks so the video fades from bright yellow to fire truck red to deep blue on a loop. Then I layered an animated video with a Stringy Sphere effect. One of the parameters on stringy sphere is wire -- this is a really cool effect that creates a wireframe sphere instead of a solid one. This also goes nicely with our 3D animation on top, and could be fun to experiment with adding a wireframe animation on top. Since both the bottom and top layer have bright colors, the result is a beautiful explosion of colors.
Next up I experimented with a wireframe palm. I decimated this plant in the Scandy app and then brought it into Meshmixer to check out different shaders. I went with a shiny light green so that it maintains some semblance of being a plant. I found that the ripple effect is really fun to morph and distort 3D objects. In this video I start by showing the original plant, then I apply the ripple effect with the spikes preset. This is the most dramatic version of this effect and makes the object almost unrecognizable. Under the parameters, I lowered the amplitudes, and then I switched over to the other preset called gentle. This looks like there is some shallow water washing over the video as it lightly tugs on the image rather than completely distorting it. Either option could be cool depending on what you’re going for.
Later on, I found a Lego in the same collection on Sketchfab. It’s actually the top of a lego box but it looks like a lego so we’ll run with it. Its extremely basic geometric shape gives a little bit more flexibility to use effects without completely distorting the shape. I added the Posterize effect to black out the background, then PolarKaleido and Fragment, using the mist preset to show dozens of faint lego reflections circling in the background. Lastly, I used Add Subtract to give it a blueish green hue.
This is really just the beginning of what you can do with 3D objects in Resolume. Incorporating 3D objects opens up another door for visual mixing, and the possibilities are truly endless.
Follow Laura on Instagram: @roostyrooster
