FS1R Formant Shaping, Part 1: Human speech

-

Introduction

I'll help you get a deeper understanding of Formant Shaping on the FS1R, split over multiple posts. 

I'll start with some background, and then dive deeper into the technical aspects.

Communication via audio

Animals

Many animals can make sounds: "bark", "oink", "tweet", and "ssssss".  It usually just means: "I'm here!"

Some can alter their calls. A dog can bark softly or loudly, to express the difference between: "I don't really appreciate this" and "I'll tear you up!". 

Some animals can pack even more information. Birds produce a whistle-like sound that can be varied in both pitch and loudness quickly.


Human speech

We can do what birds to, to a certain extent: blow air from our longs and vary the pitch.  
Birds outsing us on the technique of producing a single frequency that varies over time. 


But we've developed something truly awesome on top of that technique.

We give the outgoing sound an extra treatment. We reshape it to damp and amplify custom frequencies, corresponding to the data that we want to encode.  So not only do we have the sound that's produced in our throats. We also have a few extra tracks with frequencies to store information in. 

And with that, we pull off something remarkable: we've found a way to encode multiple streams of information in parallel into a single audio stream. Eat that, birds! 

The resonant frequencies that we produce after our throat are called formants. 



Sending data encoded in frequencies has great advantages for us;
  • We can send more data in parallel. 
    • In the same way a radio has multiple channels, we emit multiple signals on different frequencies. 
  • It's fault-tolerant.
    • The same information is repeated many times before the next package is sent. So missing a packet is not that big of a deal.
    • Using the multiple signals together allows for some error corrections to be performed in our brains. 
  • Our ears, brains, throats, and mouths are all optimized for this way of exchanging information.

The "protocol"

We don't process speech in the time domain, like a sample. That's not how our ears are built, and it's not how our brain can process audio.

Instead, we process information in the frequency domain. And we're not even processing all frequencies with the same attention. We just process the peak frequencies. 


A specific combination of boosted frequencies can be interpreted as a vowel (think "ooh", "aah", or "eee").

The human speech "protocol" also allows for other sounds to be mixed in. Sounds that don't have a very specific frequency, and are more noise-like. Like the letter S. Or sudden noisy bursts, like the P.

To have intelligible speech, all you need is:
  • 2 to 3 frequency tracks
  • some noise-like portions
This is universal for all human languages. Language and meaning are built on top of these building blocks.

The FS1R neatly condenses that model into an engine that can produce exactly what's needed (and more!).


Next time, we'll see how humans produce speech, and explain some of the FS1R terminology

Comments

Post a Comment

Popular posts from this blog

FS1R Formant Shaping, Part 2: Human formants and terms

-
Image
  What is a formant anyway? To truly understand the FS1R, you need some background information, and you need to understand some of the terminology. Let's follow the journey that air makes when you speak. When you speak, you blow air out of your lungs.  When you close your  vocal cords  a bit, the air makes causes vibrations.  The more you open the folds: the more distance they need to cross to open/close the longer it takes to flap back and forth the lower the frequency that the chords will flap the lower the tone it produces for humans, this flapping happens about 100 to 300 times per second (100-300Hz) Yamaha calls this the  Fundamental Frequency This rhythmic oscillation produces a somewhat boring "buzzing" kind of sound. The buzz consists of short bursts of sound that are rich with overtones. Each burst fades out when the vocal folds close On the way out, the air passes your throat, tongue, checks, lips, nose and teeth.  By moving those strategical...
Read more

FS1R Voices and DX7

-
Image
As an FS1R user, you probably know this already: There are 1408 voices in ROM. They're split over 11 banks (A - K) , with 128 voices each. ROM From the outside, all of these voices are just "voices".  But did you know that they're not all stored in the same way in ROM? 2 banks (A, B) are stored as native FS1R voices (608 bytes each) .  The same as the bulk dump format, but without sysex header/footer.  That's 152KB for 256 voices . 9 banks (C - K) are stored in native DX7 VCED format (155 bytes each) .  This is the format that a real DX7 can emit/consume directly. That's 174KB for 1152 voices . Most voices are stored in DX7 format.  The FS1R converts these on the fly when you select a voice. When you do a bulk-dump, you get a converted patch. The FS1R also accepts these DX7 VCED patches via MIDI . The FS1R also accepts DX7 ACED via MIDI.  ACED is an extension to VCED, adding stuff like pitch envelop scaling. Note that there is no ACED in the FS1R ...
Read more