Background and inspirations:
Remember the hiltworks project from years ago?HiltWorks Project Demo
Well, ever since I found that old thread, I've been trying to do something similar.
This guy is also doing something similar, but from what I can tell, he's synthesizing his hum. Lightsaber Tuning - YouTube
And just yesterday in Profezzorn's thread Lolwel21 came up with an interesting idea for smoothly interpolating between the hum and a looping swing sound.Announcing the Teensy Saber open source sound board
This all ties into what I'm about to describe. I had the basics of this working before Christmas, but have been holding onto it, because like any artist, I'm always wanting to make things perfect before showing them to anyone.Theory:
The actual lightsaber swing sounds in the movies are just Ben Burtt waving a microphone in front of a speaker playing the hum. There are two main effects you get form this:
- You get a small doppler shift of the sound (2 to 3 semitones up or down)
- You get a volume change as the microphone nears or retreats from the speaker
When I talk about pitch shifting, I'm talking about not just changing the sample rate, but the time-invariant pitch shifting where the length of the sound stays the same. While real-time pitch-shifting is elusive on the little MCUs we use, we can fake this by playing 3 sounds at once.
Sound 1: The original looping hum from your font (main)
Sound 2: The original looping hum, but pitch shifted a few semitones higher (high)
Sound 3: The original looping hum, but pitch shifted a few semitones lower (low)
I play all three sounds at once, but the relative gains on each are controlled by the motion of the saber. When the saber is still, you have the main hum at normal gain, and the other two have gain of 0. We want the relative volumes of the two pitch-shifted swing sounds to increase according to how hard we're swinging the saber. But how do we do this?Observations:
First, here are a few observations I noticed when watching the movies:
- Often a high-pitched swing will be followed by a lower-pitched swing. This makes sense. When swinging the microphone towards or away from the speaker, Ben Burtt can't keep swinging in the same direction. He has to swing towards the microphone and then away from it.
- There is often an effect where a single swing goes low-high-low, or high-low-high (in terms of pitch). This is the case when swinging the microphone across a speaker.
- Sometimes the swing of the saber doesn't actually change pitch in the movies, only volume. As far as I can discern, this is likely the case where the microphone was nearing the speaker without moving quickly enough to warrant a noticeable doppler shift.
- Multiple swings in the same direction don't always produce the same pitched swing sound. This means you can't just say swinging left always lerps towards the high sound, or swinging right always lerps towards the low sound.
So, how do we mimic these observations?
The first approach I tried was to have a single secondary track and increase the gain on one of the sounds (and decrease the gain on the main hum accordingly) in relation to the gyro readings. However, this means you are stuck with one swing sound, and no real variance in pitch. I believe this is similar to the approach Lolwel21 described.
I also tried having a single secondary track that blended various high-pitched and low-pitched hums over time, and then increased the gain on that track with the gyro readings, but that just sounded rythmic and throbbing when you swung during swings due to the premixed swing track.
That's where the 3 track method came into play. Part of the method I came up with is to treat the axes of the gyro independently. Each axis still contributes to both the high and low pitched hum. This will give you a series of weights for each of the pitch-shifted hums. The behavior of this is that for most cases you get the high-low alternation found in observations #1 and #2. The volume is controlled by the magnitude of the total angular velocity and is independent of the weights of the pitch shifted sounds. Because of this, there will be times where you have a swing, but the weights for each of the pitch-shifted hums ends up being zero. This solves the case where a swing may not affect the pitch (observation #3), but only the volume. Remember that we're modifying volume based on total angular velocity.
Great, but what about observation #4 above? With the method I just described, the saber would behave exactly the same for each identical swing. We don't want that. We want variance.
One solution is to allow the basis that you're using to measure the angular velocity to drift or rotate by itself. Right now, when you take gyro readings, you're taking them in the frame of reference of the gyroscope. X is x, y is y, z is z. If you change the frame of reference over some period (I found 2 or 3 seconds to work well for me), you get a nice variation in the swing pitch. This is especially cool when you are spinning and the effect varies and drifts over time.Other considerations:
- Obviously, only polyphonic fonts work with this method.
- You can use the hum from another font as the pitch-shifted hums for more interesting effects. Be careful with this as it can also sound absolutely awful.
- Clipping is a huge issue here. For dark-side fonts, this is OK, as they're supposed to be grittier, but for light-side fonts, I use a pass to increase the power of the pitch-shifted hum without modifying the absolute magnitude. It's the same thing they do for pop music. I suggest having good control over your secondary stream volume per-font.
I will post a video later once it uploads to youtube.