Academia.eduAcademia.edu

Figure from:Digital Audio Effects

See full PDFdownloadDownload figure
Figur 6: Periodic/stochastic separation of an audio signal: (a) source signal; (b) periodic part; (c) stochastic part

Figure 42 ur 6: Periodic/stochastic separation of an audio signal: (a) source signal; (b) periodic part; (c) stochastic part

Related Figures (144)

Figure 1: A sounding metal cube: sound synthesis is performed for excitation on 4 different corners and forces normal to one pair of cube faces (a); the power spectrum of the emitted sounds is given (b).
Figure 1: A sounding metal cube: sound synthesis is performed for excitation on 4 different corners and forces normal to one pair of cube faces (a); the power spectrum of the emitted sounds is given (b).
Figure 2: A sounding metal bowl: sound synthesis is performed for excitation on 3 specific locations on the surface.
Figure 2: A sounding metal bowl: sound synthesis is performed for excitation on 3 specific locations on the surface.
Ficure 4: An example of a complex geometry that can be handled with our method. The thin blade causes problems with traditional tetrahedralization methods. Figure 3: Sound synthesis with a modal approach using classical tetrahedralization with 822 modes (green) and our method with a 6 x 6 x 6 hexahedral FEM resolution, leading to 891 modes (blue): power spectrum of the sounds emitted when impacting at the 3 different locations shown in Figure 2.
Ficure 4: An example of a complex geometry that can be handled with our method. The thin blade causes problems with traditional tetrahedralization methods. Figure 3: Sound synthesis with a modal approach using classical tetrahedralization with 822 modes (green) and our method with a 6 x 6 x 6 hexahedral FEM resolution, leading to 891 modes (blue): power spectrum of the sounds emitted when impacting at the 3 different locations shown in Figure 2.
TABLE 1: Computation times in seconds and memory usage in megabytes for different grid resolutions. Computation times are given for the different steps of the calculation: discretization and computation of mass and stiffness matrices (T1), eigenvalues extraction (T2), and gains computation (T3).
TABLE 1: Computation times in seconds and memory usage in megabytes for different grid resolutions. Computation times are given for the different steps of the calculation: discretization and computation of mass and stiffness matrices (T1), eigenvalues extraction (T2), and gains computation (T3).
Figure 5: The power spectrum of the sounds resulting from impacts at the 3 different locations on the body of the object (top) and on the 3 different locations on the wing (bottom). Note that the audible response is different based on where the object is hit.
Figure 5: The power spectrum of the sounds resulting from impacts at the 3 different locations on the body of the object (top) and on the 3 different locations on the wing (bottom). Note that the audible response is different based on where the object is hit.
excitation which supplies the energy to the sound system contributes to a great extent to the fine details of the resulting sounds. Excitation signals may be produced by various ways loading recorded sound samples, using realtime signal: coming from live soundcard inputs, connecting the output of other audio applications with Pure Data through a sounc server. to tune the resulting sounds for the desired effect. For instance, the size of the geometry can be modified as different dimensions could be preferred for rendering sounds in a particular scenario. The mesh geometry is loaded in Alias\ Wavefront *.obj format, and we use Blender to apply seometrical transformations in order to test how it affects the rendering of the resulting sounds.
excitation which supplies the energy to the sound system contributes to a great extent to the fine details of the resulting sounds. Excitation signals may be produced by various ways loading recorded sound samples, using realtime signal: coming from live soundcard inputs, connecting the output of other audio applications with Pure Data through a sounc server. to tune the resulting sounds for the desired effect. For instance, the size of the geometry can be modified as different dimensions could be preferred for rendering sounds in a particular scenario. The mesh geometry is loaded in Alias\ Wavefront *.obj format, and we use Blender to apply seometrical transformations in order to test how it affects the rendering of the resulting sounds.
Figur 7: Interface for sound design. After having loaded the modal data and the corresponding mesh geometry, the user can experiment the modal sounds when exciting the object surface at different locations. Excitation signals may be loaded as recorded sound samples or realtime tracked from live soundcard inputs.
Figur 7: Interface for sound design. After having loaded the modal data and the corresponding mesh geometry, the user can experiment the modal sounds when exciting the object surface at different locations. Excitation signals may be loaded as recorded sound samples or realtime tracked from live soundcard inputs.
Figure 1: The custom-built AC30 tube guitar amplifier (photo by Ari Viitala).
Figure 1: The custom-built AC30 tube guitar amplifier (photo by Ari Viitala).
Figure 2: Dialog box for creating the analysis signal.
Figure 2: Dialog box for creating the analysis signal.
Ficurs 3: A dialog box for analyzing the system responses.
Ficurs 3: A dialog box for analyzing the system responses.
Figure 4: Graphical user interface for manual alignment of the analysis signal.
Figure 4: Graphical user interface for manual alignment of the analysis signal.
Ficure 5: Response spectra to a 1 kHz sinusoid for the real AC30 (left column), the Top30 unit of the TH1 software plugin (middle column), and the FOX ACS-45 unit of the ReValver software plugin (right column). The user adjustable gain knob was first set to 9 o’clock (top row), then to 12 o'clock (middle row), and finally to full (bottom row). As expected, the real AC30 has a higher noise floor than the digital versions.
Ficure 5: Response spectra to a 1 kHz sinusoid for the real AC30 (left column), the Top30 unit of the TH1 software plugin (middle column), and the FOX ACS-45 unit of the ReValver software plugin (right column). The user adjustable gain knob was first set to 9 o’clock (top row), then to 12 o'clock (middle row), and finally to full (bottom row). As expected, the real AC30 has a higher noise floor than the digital versions.
Figure 6: Transient analysis waveforms for the real AC30 (left column), the TH1 AC30 (middle column), and the ReValver AC30 (right column). The input signal is illustrated with a dashed line, while the solid line denotes the measured response. The gain knob was first set to 9 o'clock (top row), then to 12 o’clock (middle row), and finally to full (bottom row).
Figure 6: Transient analysis waveforms for the real AC30 (left column), the TH1 AC30 (middle column), and the ReValver AC30 (right column). The input signal is illustrated with a dashed line, while the solid line denotes the measured response. The gain knob was first set to 9 o'clock (top row), then to 12 o’clock (middle row), and finally to full (bottom row).
Figure 7: Logsweep analysis of the real AC30 (left column), the TH1 AC30 (middle column), and the ReValver AC30 (right column). The gain knob was first set to 9 o'clock (top row), then to 12 o'clock (middle row), and finally to full (bottom row). The numbered curves represent the order of the distortion, so that curve 1 corresponds to the linear magnitude response, curve 2 to the 2nd order harmonic distortion, curve 3 to the 3rd order harmonic distortion, and so forth. The 6th order and higher distortion components are not illustrated.
Figure 7: Logsweep analysis of the real AC30 (left column), the TH1 AC30 (middle column), and the ReValver AC30 (right column). The gain knob was first set to 9 o'clock (top row), then to 12 o'clock (middle row), and finally to full (bottom row). The numbered curves represent the order of the distortion, so that curve 1 corresponds to the linear magnitude response, curve 2 to the 2nd order harmonic distortion, curve 3 to the 3rd order harmonic distortion, and so forth. The 6th order and higher distortion components are not illustrated.
Figure 8: Intermodulation distortion analysis of the real AC30 (left column), the TH1 AC30 (middle column), and the ReValver AC30 (right column). The gain knob was first set to 9 o’clock (top row), then to 12 o’clock (middle row), and finally to full (bottom row). The solid line illustrates the amount of dynamic intermodulation distortion, while the dashed line denotes the amount of static intermodulation distortion, both as a function of the normalized input signal level.
Figure 8: Intermodulation distortion analysis of the real AC30 (left column), the TH1 AC30 (middle column), and the ReValver AC30 (right column). The gain knob was first set to 9 o’clock (top row), then to 12 o’clock (middle row), and finally to full (bottom row). The solid line illustrates the amount of dynamic intermodulation distortion, while the dashed line denotes the amount of static intermodulation distortion, both as a function of the normalized input signal level.
Figure 9: Aliasing analysis for the real AC30 (left column), the TH1 AC30 (middle column), and the ReValver AC30 (right column), when a 5 kHz sine is used as input. The gain knob was first set to 9 o'clock (top row), then to 12 o’clock (middle row), and finally to full (bottom row). In the figures, the circles denote the peaks of the harmonic components, while the solid line stands for aliased signal components and noise. The dashed line illustrates an auditory spectrum estimate. In both software plugins (middle and right column), the aliased components greatly exceed the estimated auditory threshold, thus suggesting that the aliasing is clearly audible. For the real AC30 (left column), the figures show that the 50 Hz power supply hum and its harmonics are also audible.
Figure 9: Aliasing analysis for the real AC30 (left column), the TH1 AC30 (middle column), and the ReValver AC30 (right column), when a 5 kHz sine is used as input. The gain knob was first set to 9 o'clock (top row), then to 12 o’clock (middle row), and finally to full (bottom row). In the figures, the circles denote the peaks of the harmonic components, while the solid line stands for aliased signal components and noise. The dashed line illustrates an auditory spectrum estimate. In both software plugins (middle and right column), the aliased components greatly exceed the estimated auditory threshold, thus suggesting that the aliasing is clearly audible. For the real AC30 (left column), the figures show that the 50 Hz power supply hum and its harmonics are also audible.
Ficure 1: Swept-sine signal x,(t) in the time domain (b), with the time length chosen according to the instantaneous frequency f;(t) (a).
Ficure 1: Swept-sine signal x,(t) in the time domain (b), with the time length chosen according to the instantaneous frequency f;(t) (a).
Figure 2: MISO model for nonlinear system identification with the input signals g,,[x(t)] and the linear filters A,(f), € [1,N].
Figure 2: MISO model for nonlinear system identification with the input signals g,,[x(t)] and the linear filters A,(f), € [1,N].
Figure 3: Generalized Polynomial Hammerstein (GPH) model (power series nonlinear model) for nonlinear system identification.
Figure 3: Generalized Polynomial Hammerstein (GPH) model (power series nonlinear model) for nonlinear system identification.
Figure 4: Circuit diagram of the home-made overdrive effect pedal.
Figure 4: Circuit diagram of the home-made overdrive effect pedal.
TaBLe 1: Effect of the number of branches N of the GPH model on the weighted harmonic distortion difference AHI-2 and on the relative computing complexity CC. Figure 5: Comparison between the real-world output of pedal 1 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fo = 500 Hz and Ay = 1 V. The GPH model is estimated using a swept sine signal with amplitude A, = 1 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
TaBLe 1: Effect of the number of branches N of the GPH model on the weighted harmonic distortion difference AHI-2 and on the relative computing complexity CC. Figure 5: Comparison between the real-world output of pedal 1 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fo = 500 Hz and Ay = 1 V. The GPH model is estimated using a swept sine signal with amplitude A, = 1 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
Figure 6: HI-2 of synthesized and real-world signals as a function of the input level (pedal 1).
Figure 6: HI-2 of synthesized and real-world signals as a function of the input level (pedal 1).
Figure 7: Comparison between the real-world output of pedal 1 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fy = 500 Hz and Ap = 0.5 V. The GPH model is estimated using a swept sine signal with amplitude A, = 1 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
Figure 7: Comparison between the real-world output of pedal 1 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fy = 500 Hz and Ap = 0.5 V. The GPH model is estimated using a swept sine signal with amplitude A, = 1 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
FrcureE 8: Comparison between the real-world output of pedal 1 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fp = 500 Hz and Ap = 0.5 V. The GPH model is estimated using a swept sine signal with amplitude A, = 0.5 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
FrcureE 8: Comparison between the real-world output of pedal 1 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fp = 500 Hz and Ap = 0.5 V. The GPH model is estimated using a swept sine signal with amplitude A, = 0.5 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
Figure 9: Comparison between the real-world output of pedal 2 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fo = 500 Hz and Ay = 1 V. The GPH model is estimated using a swept sine signal with amplitude A, = 1 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
Figure 9: Comparison between the real-world output of pedal 2 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fo = 500 Hz and Ay = 1 V. The GPH model is estimated using a swept sine signal with amplitude A, = 1 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
Ficure 10: Comparison between the real-world output of pedal 2 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fp = 500 Hz and Ap = 0.5V. The GPH model is estimated using a swept sine signal with amplitude A; = 1 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
Ficure 10: Comparison between the real-world output of pedal 2 (blue-dashed) and the GPH model-based output (red-solid) in time (a) and frequency (b) domains, for a sine wave excitation with fp = 500 Hz and Ap = 0.5V. The GPH model is estimated using a swept sine signal with amplitude A; = 1 V. For the sake of clarity, the output of the real-world device in the frequency domain is shifted to the right.
Figure 12: Impulse responses 4),(t) of the first branch of the nonlinear model (N = 7) depicted for different input levels (pedal 1),
Figure 12: Impulse responses 4),(t) of the first branch of the nonlinear model (N = 7) depicted for different input levels (pedal 1),
Figure 11: HI-2 of synthesized and real-world signals as a function of the input level (pedal 2).
Figure 11: HI-2 of synthesized and real-world signals as a function of the input level (pedal 2).
Frcure 13: Impulse responses aj,(t) of the first branch of the nonlinear model (N = 7) depicted for different input levels (pedal 2).
Frcure 13: Impulse responses aj,(t) of the first branch of the nonlinear model (N = 7) depicted for different input levels (pedal 2).
Ficure 1: An example kick drum recording, (a) is a noisy microphone signal which includes kick drum and bleed, (b) shows the amplitude envelope of the kick drum contained within the noisy signal, and (c) and (d) show the amplitude envelope of the bleed contained within the noisy signal. Part (d) has reduced limits on the y-axis to show cymbals and hi-hats in the bleed signal.
Ficure 1: An example kick drum recording, (a) is a noisy microphone signal which includes kick drum and bleed, (b) shows the amplitude envelope of the kick drum contained within the noisy signal, and (c) and (d) show the amplitude envelope of the bleed contained within the noisy signal. Part (d) has reduced limits on the y-axis to show cymbals and hi-hats in the bleed signal.
Ficure 2: Approximations to the kick drum and bleed signals, (a) contains the noisy signal y,,, (b) contains the synthesized clean kick drum signal y,, (c) contains the component of the signal attributed to bleed y,, and (d) shows the correlation of the spectral power of each window with the spectral power of the clean kick drum signal. The correlation threshold is identified by the dotted line.
Ficure 2: Approximations to the kick drum and bleed signals, (a) contains the noisy signal y,,, (b) contains the synthesized clean kick drum signal y,, (c) contains the component of the signal attributed to bleed y,, and (d) shows the correlation of the spectral power of each window with the spectral power of the clean kick drum signal. The correlation threshold is identified by the dotted line.
TABLE 1: Peak signal level in the bleed regions identified by f, and f for a range of relative bleed strengths. The hold time gives what appears to be the most unintuitive results. For signals with relative bleed strengths of +0 dB, +2 dB, and +4 dB, the hold time remains roughly constant at around 40ms. The signal which has a bleed strength of +6 dB has a far lower hold time when the required bleed reduction is time will depend on kick drum signal is the degree to whic fluctuating about t arge, and shows a sudden increase in hold time when Opleeq > —20dB. The value o f the hold ope of the d. If there h the enve he thresho are substantial fluctuations a longer hold time is required. The hold time is determined using the initial e the threshold. Signa s with different re have different initia S fl t fl stimate of ative bleed strengths threshold estimates. Evidently for the ignal with a bleed strength of +6dB, there are minimal uctuations in the envelope of the kick d initial threshold estimate when the required bleed reduction is large. When the required bleed red he initial threshold estimate is lower, rum signal about the uction is decreased, and there are more uctuations in the envelope of the kic k drum signal about it. A longer hold time is therefore needed.
TABLE 1: Peak signal level in the bleed regions identified by f, and f for a range of relative bleed strengths. The hold time gives what appears to be the most unintuitive results. For signals with relative bleed strengths of +0 dB, +2 dB, and +4 dB, the hold time remains roughly constant at around 40ms. The signal which has a bleed strength of +6 dB has a far lower hold time when the required bleed reduction is time will depend on kick drum signal is the degree to whic fluctuating about t arge, and shows a sudden increase in hold time when Opleeq > —20dB. The value o f the hold ope of the d. If there h the enve he thresho are substantial fluctuations a longer hold time is required. The hold time is determined using the initial e the threshold. Signa s with different re have different initia S fl t fl stimate of ative bleed strengths threshold estimates. Evidently for the ignal with a bleed strength of +6dB, there are minimal uctuations in the envelope of the kick d initial threshold estimate when the required bleed reduction is large. When the required bleed red he initial threshold estimate is lower, rum signal about the uction is decreased, and there are more uctuations in the envelope of the kic k drum signal about it. A longer hold time is therefore needed.
Ficure 3: Automatic noise gate flow chart.
Ficure 3: Automatic noise gate flow chart.
Figure 4: Kick drum recording before and after gating, (a) before gating, and (b) after gating, with dyteeq = —60 dE
Figure 4: Kick drum recording before and after gating, (a) before gating, and (b) after gating, with dyteeq = —60 dE
Ficure 5: Noise gate parameter values after optimization, plotted against the required reduction in bleed (dyiceq as defined in Figure 3). Part (a) shows threshold, (b) shows release time, (c) shows hold time and (d) shows SAR. Results are plotted for a number of relative bleed strengths identified by, ©: +0 dB, ©: +2 dB, «: +4 dB, x: +6 dB.
Ficure 5: Noise gate parameter values after optimization, plotted against the required reduction in bleed (dyiceq as defined in Figure 3). Part (a) shows threshold, (b) shows release time, (c) shows hold time and (d) shows SAR. Results are plotted for a number of relative bleed strengths identified by, ©: +0 dB, ©: +2 dB, «: +4 dB, x: +6 dB.
where hx (n)—the impulse response of the band-pass filter with passband [Fx_1, Fx], s4(n)—bandlimited output signal. The impulse response can be written in the following way: 3.1. Instantaneous Harmonic Analysis of Nonstationary Har- monic Components. The proposed analysis method is based on the filtering technique that provides direct parameters estimation [8]. In voiced speech harmonic components are spaced in frequency domain and each component can be limited thereby a narrow frequency band. Therefore harmonic components can be separated within the analysis frame by filters with nonoverlapping bandwidths. These considerations point to the applicability and effectiveness of the filtering approach to harmonic analysis. The signal s(n) is represented as a sum of bandlimited cosine functions with instantaneous amplitude, phase, and frequency. It is assumed that harmonic components are spaced in frequency domain so that each component can be limited by a narrow freq uency band. The harmonic components can be separated within the analysis frame by filters with nonoverlapping bandwidths. Let us denote the number of cosines L and frequency separation borders (in Hz) Fp < Fp < ::: < Fi, where Fo = 0, Fp = F,/2. The given signal s(n) can be represented as its convolution with the impulse response of the ideal low-pass filter h(n):
where hx (n)—the impulse response of the band-pass filter with passband [Fx_1, Fx], s4(n)—bandlimited output signal. The impulse response can be written in the following way: 3.1. Instantaneous Harmonic Analysis of Nonstationary Har- monic Components. The proposed analysis method is based on the filtering technique that provides direct parameters estimation [8]. In voiced speech harmonic components are spaced in frequency domain and each component can be limited thereby a narrow frequency band. Therefore harmonic components can be separated within the analysis frame by filters with nonoverlapping bandwidths. These considerations point to the applicability and effectiveness of the filtering approach to harmonic analysis. The signal s(n) is represented as a sum of bandlimited cosine functions with instantaneous amplitude, phase, and frequency. It is assumed that harmonic components are spaced in frequency domain so that each component can be limited by a narrow freq uency band. The harmonic components can be separated within the analysis frame by filters with nonoverlapping bandwidths. Let us denote the number of cosines L and frequency separation borders (in Hz) Fp < Fp < ::: < Fi, where Fo = 0, Fp = F,/2. The given signal s(n) can be represented as its convolution with the impulse response of the ideal low-pass filter h(n):
The required instantaneous parameters can be calculated using expressions (18). The frequency-modulated filter has a warped band pass, aligned to the given frequency contour F (n) that provides adequate analysis of periodic compo- nents with rapid frequency alterations. This approach is an alternative to time warping that is used in speech analysis [11]. In Figure 1 an example of parameters estimation is shown. The frequency contour of the harmonic component can be covered by the filter band pass specified by the centre frequency contour F*(n) and the bandwidth 2Fk. Ficure 1: Frequency-modulated analysis filter N = 512.
The required instantaneous parameters can be calculated using expressions (18). The frequency-modulated filter has a warped band pass, aligned to the given frequency contour F (n) that provides adequate analysis of periodic compo- nents with rapid frequency alterations. This approach is an alternative to time warping that is used in speech analysis [11]. In Figure 1 an example of parameters estimation is shown. The frequency contour of the harmonic component can be covered by the filter band pass specified by the centre frequency contour F*(n) and the bandwidth 2Fk. Ficure 1: Frequency-modulated analysis filter N = 512.
Ficure 3: Minimal bandwidth of analysis filter. Ficure 2: Instantaneous amplitude estimation accuracy.
Ficure 3: Minimal bandwidth of analysis filter. Ficure 2: Instantaneous amplitude estimation accuracy.
Figure 4: Instantaneous frequency estimation error.
Figure 4: Instantaneous frequency estimation error.
Ficure 5: Sinusoidal parameters estimation using analysis filters: (a) initial frequency partition; (b) frequency partition after second iteration.
Ficure 5: Sinusoidal parameters estimation using analysis filters: (a) initial frequency partition; (b) frequency partition after second iteration.
Figure 7: Harmonic analysis of speech. where 67;—energy of the estimated harmonic part and o3,;— energy of the estimation error (energy of the difference between source and estimated harmonic parts). The signals were analyzed using the proposed technique and STFT- based harmonic transform method [10]. During analysis the same frame length was used (64ms) and the same window function (Hamming window). In both methods, it was assumed that the fundamental frequency contour is known and that frequency trajectories of the harmonics are integer multiplies of the fundamental frequency. The results, reported in Table 1 show that the measured SNRy values decrease with HNR values. However, for nonstationary signals, the proposed technique provides higher SNR# values even when HNR is low.
Figure 7: Harmonic analysis of speech. where 67;—energy of the estimated harmonic part and o3,;— energy of the estimation error (energy of the difference between source and estimated harmonic parts). The signals were analyzed using the proposed technique and STFT- based harmonic transform method [10]. During analysis the same frame length was used (64ms) and the same window function (Hamming window). In both methods, it was assumed that the fundamental frequency contour is known and that frequency trajectories of the harmonics are integer multiplies of the fundamental frequency. The results, reported in Table 1 show that the measured SNRy values decrease with HNR values. However, for nonstationary signals, the proposed technique provides higher SNR# values even when HNR is low.
then any timbre modification can be expressed as a con- version function C(E) that transforms the source envelope matrix E into a new matrix E: As long as described pitch shifting does not change spectral envelope of the source signal and keeps relative phases of the harmonic components, the processed signal has a natural sound with completely new intonation. The timbre of speakers voice is defined by the spectral envelope function E(n, f ). If we consider the envelope function as a matrix
then any timbre modification can be expressed as a con- version function C(E) that transforms the source envelope matrix E into a new matrix E: As long as described pitch shifting does not change spectral envelope of the source signal and keeps relative phases of the harmonic components, the processed signal has a natural sound with completely new intonation. The timbre of speakers voice is defined by the spectral envelope function E(n, f ). If we consider the envelope function as a matrix
recording. Since only pitch contour is changed, the source voice maintains its identity. The output signal however is damped in regions, where the energy of the reference signal Ficure 8: Periodic/stochastic separation of an audio signal: (a) source signal; (b) periodic part; (c) stochastic part.
recording. Since only pitch contour is changed, the source voice maintains its identity. The output signal however is damped in regions, where the energy of the reference signal Ficure 8: Periodic/stochastic separation of an audio signal: (a) source signal; (b) periodic part; (c) stochastic part.
is low in order to provide proper synchronization with accompaniment. The reference signal is shown in Figure 9, it is a recorded male vocal. The recording was made in a studio
is low in order to provide proper synchronization with accompaniment. The reference signal is shown in Figure 9, it is a recorded male vocal. The recording was made in a studio
TABLE 1: Results of synthetic speech analysis.
TABLE 1: Results of synthetic speech analysis.
Signal 4— fo(n) changes from 150 to 220 Hz at a rate of 0.1 Hz/ms, variable harmonic amplitudes that model sequence of vowels, harmonic frequencies deviate from integer multiplies of fo(n) on 10 Hz
Signal 4— fo(n) changes from 150 to 220 Hz at a rate of 0.1 Hz/ms, variable harmonic amplitudes that model sequence of vowels, harmonic frequencies deviate from integer multiplies of fo(n) on 10 Hz
FicureE 1: General diagram of a cross-adaptive device using side chain processing with feature accumulation.
FicureE 1: General diagram of a cross-adaptive device using side chain processing with feature accumulation.
Figure 2: Quasiflat frequency response bandpass filter bank (Type A filter bank for K = 8). (a) Filter bank consisting of a set of eight second order band-pass IIR Biquadratic filters with center frequencies as follow: 100Hz, 400Hz, 1kHz, 2.5kHz, 5kHz, 7.5kHz, 10kHz, and 15000 kHz. (b) Combined response of the filter bank.
Figure 2: Quasiflat frequency response bandpass filter bank (Type A filter bank for K = 8). (a) Filter bank consisting of a set of eight second order band-pass IIR Biquadratic filters with center frequencies as follow: 100Hz, 400Hz, 1kHz, 2.5kHz, 5kHz, 7.5kHz, 10kHz, and 15000 kHz. (b) Combined response of the filter bank.
Figure 3: Lowpass filter decomposition filter bank (Type B filter bank for K = 8). (a) Filter bank comprised of a set of second order low-pass IIR Biquadratic filters with cutoff frequencies as follows: 35 Hz, 80 Hz, 187.5 Hz, 375 Hz, 750 Hz, 1.5 kHz, 3 kHz, and 6 kHz. (b) Combined response of the filter bank. All gains have been set to have a maximum peak value of 0 dBs.
Figure 3: Lowpass filter decomposition filter bank (Type B filter bank for K = 8). (a) Filter bank comprised of a set of second order low-pass IIR Biquadratic filters with cutoff frequencies as follows: 35 Hz, 80 Hz, 187.5 Hz, 375 Hz, 750 Hz, 1.5 kHz, 3 kHz, and 6 kHz. (b) Combined response of the filter bank. All gains have been set to have a maximum peak value of 0 dBs.
The following equation gives the panning space location of the ith source residing in the kth filter: where P(i, k) is the ith available panning step in the kth filter, i ranges from 1 to Nx and P(i,k) = 0.5 corresponds to a center panning position.
The following equation gives the panning space location of the ith source residing in the kth filter: where P(i, k) is the ith available panning step in the kth filter, i ranges from 1 to Nx and P(i,k) = 0.5 corresponds to a center panning position.
Figure 4: Analysis block diagram for one input channel.
Figure 4: Analysis block diagram for one input channel.
Figure 5: Block diagram of the automatic panner constrained control rules algorithm for K input channels.
Figure 5: Block diagram of the automatic panner constrained control rules algorithm for K input channels.
Figure 6: Results of automatic panning based on the proposed design. The test inputs were 12 sinusoids with amplitude equal to one and the following frequencies: f,| = 125Hz, fp = 5kHz, fs = 15kHz, fs = 5kHz, fs = 20kHz, fs = 5kHz, fp = 15kHz, fg = 20 kHz, fo = 15 kHz, fio = 15 kHz, fi: = 10 kHz, and fi, = 125 Hz.
Figure 6: Results of automatic panning based on the proposed design. The test inputs were 12 sinusoids with amplitude equal to one and the following frequencies: f,| = 125Hz, fp = 5kHz, fs = 15kHz, fs = 5kHz, fs = 20kHz, fs = 5kHz, fp = 15kHz, fg = 20 kHz, fo = 15 kHz, fio = 15 kHz, fi: = 10 kHz, and fi, = 125 Hz.
Ficure 7: Double blind panning quality evaluation test interface.
Ficure 7: Double blind panning quality evaluation test interface.
Ficure 8: Summarized results for the subjective evaluation. The first two tests were references (comparing stereo against monaural, and comparing identical files), and the remaining questions compared the two proposed auto-panning methods against each other and against expert and non-expert mixes. 95% confidence intervals are provided.
Ficure 8: Summarized results for the subjective evaluation. The first two tests were references (comparing stereo against monaural, and comparing identical files), and the remaining questions compared the two proposed auto-panning methods against each other and against expert and non-expert mixes. 95% confidence intervals are provided.
Figure 9: Distribution of sources among filters for methods A and B for the same song (Pw = 0.059)
Figure 9: Distribution of sources among filters for methods A and B for the same song (Pw = 0.059)
Ficure 10: Panning space distribution histograms (Pw = 0.059).
Ficure 10: Panning space distribution histograms (Pw = 0.059).
TABLE 1: Double blind panning quality evaluation table.
TABLE 1: Double blind panning quality evaluation table.
Figure 1: The specialized RRP (a) and the set of rays passing through the reference reflector in the (m, q) domain (visibility region from the reference reflector) (b).
Figure 1: The specialized RRP (a) and the set of rays passing through the reference reflector in the (m, q) domain (visibility region from the reference reflector) (b).
FrcureE 2: Ambiguity in the occlusion between two nonoriented reflectors s. and s3. For some rays (€.g., fq) s2 occludes s3. For other rays (e.g., rp) $3 occludes sy.
FrcureE 2: Ambiguity in the occlusion between two nonoriented reflectors s. and s3. For some rays (€.g., fq) s2 occludes s3. For other rays (e.g., rp) $3 occludes sy.
Ficure 3: Beams traced from the source location (a) and the corresponding beam-tree (b).
Ficure 3: Beams traced from the source location (a) and the corresponding beam-tree (b).
Ficure 4: (a) Beam subdivision performed in the (m,q) domain for the bundle of rays corresponding to the reflection of the beam b3 of Figure 3. (b) Corresponding subbeams in the geometric domain.
Ficure 4: (a) Beam subdivision performed in the (m,q) domain for the bundle of rays corresponding to the reflection of the beam b3 of Figure 3. (b) Corresponding subbeams in the geometric domain.
Figure 5: Path-tracing from source S to the receiver R. The beam-tree on the right-hand side is the same as in Figure 3
Figure 5: Path-tracing from source S to the receiver R. The beam-tree on the right-hand side is the same as in Figure 3
Figure 6: Tracing paths in the visibility diagram: the parameters of the outgoing ray are found by means of the intersection of the dual of the beam (a segment) and of the receiver (a line). Paths falling in beams limited by a reflector, also (2) must hold. The conditions in (1) and (possibly) (2) are tested for all the beams. However, if R falls onto by, then we know that it cannot fall in other beams that share the virtual source of b;.. This speeds up the path-tracing process a great deal. As an example of the tracing process, consider the situation shown in Figure 5. Here S and R are not in the line of sight, but a reflective path exists through the reflection from s3. First- order beams are traced directly in the geometric domain, as done in [12], therefore the presence of R in beams originating directly from S is tested directly in the geometric domain. Let us now test the presence of R in the reflected beams emerging from b3 (see Figure 3). The intersection between
Figure 6: Tracing paths in the visibility diagram: the parameters of the outgoing ray are found by means of the intersection of the dual of the beam (a segment) and of the receiver (a line). Paths falling in beams limited by a reflector, also (2) must hold. The conditions in (1) and (possibly) (2) are tested for all the beams. However, if R falls onto by, then we know that it cannot fall in other beams that share the virtual source of b;.. This speeds up the path-tracing process a great deal. As an example of the tracing process, consider the situation shown in Figure 5. Here S and R are not in the line of sight, but a reflective path exists through the reflection from s3. First- order beams are traced directly in the geometric domain, as done in [12], therefore the presence of R in beams originating directly from S is tested directly in the geometric domain. Let us now test the presence of R in the reflected beams emerging from b3 (see Figure 3). The intersection between
the line ¢ Dy = mx + ay we can easily conclude that R is not occluded by s; if the distance between S and R is smaller than the distance between S and the intersection of the (7m, q) ray with s;, which means that:
the line ¢ Dy = mx + ay we can easily conclude that R is not occluded by s; if the distance between S and R is smaller than the distance between S and the intersection of the (7m, q) ray with s;, which means that:
Figure 7: Diffusion phenomenon: a planar wavefront falls onto a rough surface from a perpendicular direction. The wavefront that bounces off the surface, resulting from a combination of secondary wavefronts, will not propagate just in the specular direction with respect to the exciting wave.
Figure 7: Diffusion phenomenon: a planar wavefront falls onto a rough surface from a perpendicular direction. The wavefront that bounces off the surface, resulting from a combination of secondary wavefronts, will not propagate just in the specular direction with respect to the exciting wave.
Figure 8: Geometric Theory of Diffraction: an acoustic source is in S. The acoustic source interacts with the obstacle, producing diffracted rays. Given the source position S, the points on the edge behave as secondary sources (e.g., P; and P3 in the figure). According to the geometrical theory of diffraction, the angle between the outgoing rays and the edge equals the angle between the incoming ray and the edge. The envelope of the outgoing rays forms a cone, known in the literature as Keller cone.
Figure 8: Geometric Theory of Diffraction: an acoustic source is in S. The acoustic source interacts with the obstacle, producing diffracted rays. Given the source position S, the points on the edge behave as secondary sources (e.g., P; and P3 in the figure). According to the geometrical theory of diffraction, the angle between the outgoing rays and the edge equals the angle between the incoming ray and the edge. The envelope of the outgoing rays forms a cone, known in the literature as Keller cone.
FicureE 9: The real source S and the receiver R face the diffusive reflector s;. Reflector s) partially occludes s, with respect to the receiver. S’ is the image-source of S mirrored over the prolongation of s;. The segments Sq and qR form a diffusive path.
FicureE 9: The real source S and the receiver R face the diffusive reflector s;. Reflector s) partially occludes s, with respect to the receiver. S’ is the image-source of S mirrored over the prolongation of s;. The segments Sq and qR form a diffusive path.
Ficure 10: Normalized geometric domain (a) and corresponding dual space (b). The lines fg and €s are the dual of the points S’ and R.
Ficure 10: Normalized geometric domain (a) and corresponding dual space (b). The lines fg and €s are the dual of the points S’ and R.
Figure 11: An example of diffractive wedge and beams departing from it. Notice that if either the source or the receiver fall outside the regions marked as I and II, then there is direct visibility, therefore diffraction can be neglected. a single edge. If the angular opening of the wedge is smaller than a and both the receiver and the source fall inside the wedge, then source and receiver are in direct visibility. Not all wedges are, therefore, worth retaining. Even if a wedge is diffractive, we can still find configurations where source and receiver are in direct visibility. When this happens, diffraction is less relevant than the direct path and we discard these diffractive paths. With reference to Figure 11, we are interested in auralizing diffraction in the two regions marked as I and II, where source and receiver are not necessarily in conditions of mutual visibility. For each of the two regions we will build a beam-tree. This selection process returns a list i = 1,...,M of diffractive wedges and their coordinates.
Figure 11: An example of diffractive wedge and beams departing from it. Notice that if either the source or the receiver fall outside the regions marked as I and II, then there is direct visibility, therefore diffraction can be neglected. a single edge. If the angular opening of the wedge is smaller than a and both the receiver and the source fall inside the wedge, then source and receiver are in direct visibility. Not all wedges are, therefore, worth retaining. Even if a wedge is diffractive, we can still find configurations where source and receiver are in direct visibility. When this happens, diffraction is less relevant than the direct path and we discard these diffractive paths. With reference to Figure 11, we are interested in auralizing diffraction in the two regions marked as I and II, where source and receiver are not necessarily in conditions of mutual visibility. For each of the two regions we will build a beam-tree. This selection process returns a list i = 1,...,M of diffractive wedges and their coordinates.
Ficure 12: When the source and the receiver fall outside the regions I and II, diffraction becomes negligible. The diffractive beam-trees will therefore cover regions | and II only.
Ficure 12: When the source and the receiver fall outside the regions I and II, diffraction becomes negligible. The diffractive beam-trees will therefore cover regions | and II only.
Figure 13: Example of a test environment used for assessing the computational efficiency of our beam tracing algorithm. The 8 m x 4m rooms are all connected together through randomly-located apertures.
Figure 13: Example of a test environment used for assessing the computational efficiency of our beam tracing algorithm. The 8 m x 4m rooms are all connected together through randomly-located apertures.
Ficure 14: Beam-tree building time for variable geometry for traditional beam tracing (circles) and visibility-based beam tracing (squares) for 10,000 beams. The proposed approach greatly out- performs traditional beam tracing especially when the number of traced beams is very large.
Ficure 14: Beam-tree building time for variable geometry for traditional beam tracing (circles) and visibility-based beam tracing (squares) for 10,000 beams. The proposed approach greatly out- performs traditional beam tracing especially when the number of traced beams is very large.
Ficure 15: Tracing time of the diffusive paths as a function of the number of walls of the environment of Figure 13.
Ficure 15: Tracing time of the diffusive paths as a function of the number of walls of the environment of Figure 13.
Ficure 16: Auralization time for diffractive paths (measured in seconds) versus the complexity (measured by the number of walls) of the environment.
Ficure 16: Auralization time for diffractive paths (measured in seconds) versus the complexity (measured by the number of walls) of the environment.
Ficure 18: Floor-map of the validation environment: the grid represents the 77 acquisition points and the loudspeaker location is marked with a dot. Figure 17: Number of diffractive paths versus the complexity of the environment (measured by the number of walls) in which experiments are done.
Ficure 18: Floor-map of the validation environment: the grid represents the 77 acquisition points and the loudspeaker location is marked with a dot. Figure 17: Number of diffractive paths versus the complexity of the environment (measured by the number of walls) in which experiments are done.
Ficure 20: Normalized energy map: (a) simulated with reflections only, (b) simulated with reflections and diffraction, (c) simulated with reflections, diffraction and diffusion, (d) measured.
Ficure 20: Normalized energy map: (a) simulated with reflections only, (b) simulated with reflections and diffraction, (c) simulated with reflections, diffraction and diffusion, (d) measured.
Figure 21: Center Time map: measured (a) simulated with reflections only, (b) simulated with reflections and diffraction, (c) simulated with reflections, diffraction and diffusion, (d) measured.
Figure 21: Center Time map: measured (a) simulated with reflections only, (b) simulated with reflections and diffraction, (c) simulated with reflections, diffraction and diffusion, (d) measured.
Figure 1: Block diagram of the direct crosstalk cancellation system for stereo loudspeakers. 2. Conventional Crosstalk Canceller It is common to use two loudspeakers in a stereo system. A block diagram of the direct implementation of crosstalk cancellation is illustrated in Figure 1 for a stereo loudspeaker system. The input binaural signals from left and right channels are given in vector form X(z) = [Xi(z),X2(z)]", and the signals received by two ears are denoted as D(z) = [Dy(z),D2(z)]". (Here signals are expressed in the Z domain.) The objective of crosstalk cancellation is to perfectly reproduce the binaural signals at the listener’s eardrums, that is, D(z) = z~4X(z), where z~“@ is the delay term, via inverting the acoustic path G(z) with the crosstalk cancellation filter H(z). Generally, the loudspeaker response should also be inverted when designing the crosstalk can- celler; however, this part can be implemented separately and thus is not considered in this paper for the convenience of analysis. G(z) and H(z) are, respectively, denoted in matrix forms as
Figure 1: Block diagram of the direct crosstalk cancellation system for stereo loudspeakers. 2. Conventional Crosstalk Canceller It is common to use two loudspeakers in a stereo system. A block diagram of the direct implementation of crosstalk cancellation is illustrated in Figure 1 for a stereo loudspeaker system. The input binaural signals from left and right channels are given in vector form X(z) = [Xi(z),X2(z)]", and the signals received by two ears are denoted as D(z) = [Dy(z),D2(z)]". (Here signals are expressed in the Z domain.) The objective of crosstalk cancellation is to perfectly reproduce the binaural signals at the listener’s eardrums, that is, D(z) = z~4X(z), where z~“@ is the delay term, via inverting the acoustic path G(z) with the crosstalk cancellation filter H(z). Generally, the loudspeaker response should also be inverted when designing the crosstalk can- celler; however, this part can be implemented separately and thus is not considered in this paper for the convenience of analysis. G(z) and H(z) are, respectively, denoted in matrix forms as
Combining G(z) and H(z), we get the global transfer function The off-diagonal items of (21) are always zeros regardless the value of T(z). This implies that the crosstalk is almost fully suppressed. However, due to the filtering effect by the diagonal items in (21), distortion will be introduced when reproducing the target signals. This is the inherent disadvantage of the single-filter structure method. The crosstalk cancellation filter is obtained from (12) and (18), with its filter length
Combining G(z) and H(z), we get the global transfer function The off-diagonal items of (21) are always zeros regardless the value of T(z). This implies that the crosstalk is almost fully suppressed. However, due to the filtering effect by the diagonal items in (21), distortion will be introduced when reproducing the target signals. This is the inherent disadvantage of the single-filter structure method. The crosstalk cancellation filter is obtained from (12) and (18), with its filter length
To find the pole coefficients vector a and the zero coefficients vector b;, i = 1,...,K, we minimize the error J and obtain that
To find the pole coefficients vector a and the zero coefficients vector b;, i = 1,...,K, we minimize the error J and obtain that
where dj, di2, do, and d22 are the transmission delays from the loudspeakers to the ears. Pa, en Caer ‘ec 7 ek. Without loss of generality, assume dj, + dy2 < diz + doi, and let A = (diy + dx) —(diot+ do). Substituting A into (29), we get
where dj, di2, do, and d22 are the transmission delays from the loudspeakers to the ears. Pa, en Caer ‘ec 7 ek. Without loss of generality, assume dj, + dy2 < diz + doi, and let A = (diy + dx) —(diot+ do). Substituting A into (29), we get
where B is a convolution matrix of size L3 x L, by cascading the vector b, and L3 = L, + L, — 1, where B* is the pseudoinverse of B, and B* is given by
where B is a convolution matrix of size L3 x L, by cascading the vector b, and L3 = L, + L, — 1, where B* is the pseudoinverse of B, and B* is given by
TaBLE 1: Parameters for the three methods: the least-squares method, the single-filter structure method, and the CAPZ method. crosstalk cancellation implementation is proportional to the crosstalk cancellation filter length, as listed in Table 1. Since Le > Ny + Ng + dmax holds for the CAPZ model, we have TABLE 2: Computational complexity of crosstalk cancellation filter estimation for the three methods: the least-squares method, the single-filter structure method, and the CAPZ method.
TaBLE 1: Parameters for the three methods: the least-squares method, the single-filter structure method, and the CAPZ method. crosstalk cancellation implementation is proportional to the crosstalk cancellation filter length, as listed in Table 1. Since Le > Ny + Ng + dmax holds for the CAPZ model, we have TABLE 2: Computational complexity of crosstalk cancellation filter estimation for the three methods: the least-squares method, the single-filter structure method, and the CAPZ method.
3.3.1. Computational Complexity of Crosstalk Cancellation Filter Estimation. From (9), (12), and (30), it is found that estimating the inverse filters h, t, and c consumes the major computation of crosstalk cancellation filter estima- tion. Thus only the computation of calculating the inverse filters is considered. Generally, the computational complexity of inverting a matrix of size N x N is O(N?), without taking advantage of matrix symmetry. The computation of estimating the inverse filters h, t, and c is closely related to the size of the matrix G, Q, and B, respectively. Supposing that the inverse filter lengths in the three methods are equal, that is, Ly = Ly = Ly = Liny, we summarize the computational complexity in Table 2 for the three methods (referring to (9), (18), and (34)). The computational complexity is calculated in terms of multiplication. For example, when the size of G is 2L; X 2Lp, the number of calculations involved in matrix multiplication is 16L7L;, and matrix inversion is O((2L;)°) (referring to (9), (10), and Table 1). Thus, the computation cost of the least-squares method is 8(O(L}) +21;,L1), as listed in Table 2. The computation cost of the other two methods can be obtained in a similar way.
3.3.1. Computational Complexity of Crosstalk Cancellation Filter Estimation. From (9), (12), and (30), it is found that estimating the inverse filters h, t, and c consumes the major computation of crosstalk cancellation filter estima- tion. Thus only the computation of calculating the inverse filters is considered. Generally, the computational complexity of inverting a matrix of size N x N is O(N?), without taking advantage of matrix symmetry. The computation of estimating the inverse filters h, t, and c is closely related to the size of the matrix G, Q, and B, respectively. Supposing that the inverse filter lengths in the three methods are equal, that is, Ly = Ly = Ly = Liny, we summarize the computational complexity in Table 2 for the three methods (referring to (9), (18), and (34)). The computational complexity is calculated in terms of multiplication. For example, when the size of G is 2L; X 2Lp, the number of calculations involved in matrix multiplication is 16L7L;, and matrix inversion is O((2L;)°) (referring to (9), (10), and Table 1). Thus, the computation cost of the least-squares method is 8(O(L}) +21;,L1), as listed in Table 2. The computation cost of the other two methods can be obtained in a similar way.
Ficure 2: Comparison of the original and modeled right ear HRTF at elevation 0°, azimuth 30°. 4.2. Performance Metrics. Two performance measures are used: the signal-to-crosstalk ratio (SCR) and the signal- to-distortion ratio (SDR) [8]. Regarding to (6), the ideal crosstalk cancellation result should be 4.3. Performance Evaluation in Symmetric Cases. In this experiment, the loudspeakers are placed in symmetric posi- tions. Three crosstalk cancellation methods are compared: the least-squares method, the single-filter structure method, and the proposed method based on CAPZ models. To be consistent with the assumption in computational complexity analysis in Section 3.3, the inverse filter lengths in the three methods are set equal, that is, Ll, = Ly; = L;. A total of 63 crosstalk cancellation systems are designed at 7 different elevations uniformly spaced between 0° and 67.5° and 9 different azimuths uniformly spaced between 5° and 45°. For each crosstalk cancellation system, various inverse filter lengths ranging from 50 to 400 samples with an interval of 50 are tested. Generally, the crosstalk cancellation performance is not quite sensitive to the delay value; however, an optimal delay value is selected for each method separately so that they can be compared in a fair condition. Since the relationship between the crosstalk cancellation and the delay z~4 shows no evident regularity, we choose the delay value experimentally. For each experiment case, the optimal delay is selected experimentally from values ranging from 50 to 400 samples with an interval of 50, ensuring that the crosstalk cancellation algorithm performs best with this optimal delay. Table 3 lists the optimal delay for the three methods at various inverse filter lengths. The regularization parameter is set empirically as 8 = 0.005 throughout the experiment. The mean value of the performance metrics over all 63 crosstalk cancellation systems is calculated.
Ficure 2: Comparison of the original and modeled right ear HRTF at elevation 0°, azimuth 30°. 4.2. Performance Metrics. Two performance measures are used: the signal-to-crosstalk ratio (SCR) and the signal- to-distortion ratio (SDR) [8]. Regarding to (6), the ideal crosstalk cancellation result should be 4.3. Performance Evaluation in Symmetric Cases. In this experiment, the loudspeakers are placed in symmetric posi- tions. Three crosstalk cancellation methods are compared: the least-squares method, the single-filter structure method, and the proposed method based on CAPZ models. To be consistent with the assumption in computational complexity analysis in Section 3.3, the inverse filter lengths in the three methods are set equal, that is, Ll, = Ly; = L;. A total of 63 crosstalk cancellation systems are designed at 7 different elevations uniformly spaced between 0° and 67.5° and 9 different azimuths uniformly spaced between 5° and 45°. For each crosstalk cancellation system, various inverse filter lengths ranging from 50 to 400 samples with an interval of 50 are tested. Generally, the crosstalk cancellation performance is not quite sensitive to the delay value; however, an optimal delay value is selected for each method separately so that they can be compared in a fair condition. Since the relationship between the crosstalk cancellation and the delay z~4 shows no evident regularity, we choose the delay value experimentally. For each experiment case, the optimal delay is selected experimentally from values ranging from 50 to 400 samples with an interval of 50, ensuring that the crosstalk cancellation algorithm performs best with this optimal delay. Table 3 lists the optimal delay for the three methods at various inverse filter lengths. The regularization parameter is set empirically as 8 = 0.005 throughout the experiment. The mean value of the performance metrics over all 63 crosstalk cancellation systems is calculated.
TaBLE 3: Optimal delay d at various inverse filter lengths (in samples) for the three methods: the least-squares method (LS), the single-filter structure method (SF), and the CAPZ method. Figure 4 shows the mean signal-to-crosstalk ratio (SCR), respectively, for the three methods with various inverse filter lengths. The horizontal axis is the inverse filter length ranging from 50 to 400 samples. The vertical axis is the mean signal- to-crosstalk ratio. Since the SCR of the SF method can be as high as 300 dB for all simulation cases, which is much higher than the levels of the other two methods (20-30 dB), its curve is left out of the picture. The SCR of the CAPZ is higher than the least-squares method. It can be seen from Figures 3 and 4 that the single-filter structure method yields the best SCR performance, while the least-squares method yields best SDR performance. On the other hand, for both SDR and SCR measures, the proposed CAPZ method yields performance that is superior to one of the reference methods, but inferior to the other reference. In a view of crosstalk cancellation, the performance of the CAPZ method is in the middle of the three methods. It can yield comparable crosstalk cancellation as the other two methods do.
TaBLE 3: Optimal delay d at various inverse filter lengths (in samples) for the three methods: the least-squares method (LS), the single-filter structure method (SF), and the CAPZ method. Figure 4 shows the mean signal-to-crosstalk ratio (SCR), respectively, for the three methods with various inverse filter lengths. The horizontal axis is the inverse filter length ranging from 50 to 400 samples. The vertical axis is the mean signal- to-crosstalk ratio. Since the SCR of the SF method can be as high as 300 dB for all simulation cases, which is much higher than the levels of the other two methods (20-30 dB), its curve is left out of the picture. The SCR of the CAPZ is higher than the least-squares method. It can be seen from Figures 3 and 4 that the single-filter structure method yields the best SCR performance, while the least-squares method yields best SDR performance. On the other hand, for both SDR and SCR measures, the proposed CAPZ method yields performance that is superior to one of the reference methods, but inferior to the other reference. In a view of crosstalk cancellation, the performance of the CAPZ method is in the middle of the three methods. It can yield comparable crosstalk cancellation as the other two methods do.
Ficure 4: Mean signal-to-crosstalk ratio (SCR) at different inverse filter lengths for the three methods: the least-squares method (LS), the single-filter structure method (SF), and the CAPZ method. (Note that the curve of the SF method is not depicted in the picture, because its SCR values can be as high as 300 dB for all simulation cases.)
Ficure 4: Mean signal-to-crosstalk ratio (SCR) at different inverse filter lengths for the three methods: the least-squares method (LS), the single-filter structure method (SF), and the CAPZ method. (Note that the curve of the SF method is not depicted in the picture, because its SCR values can be as high as 300 dB for all simulation cases.)
TABLE 4: Mean crosstalk cancellation performance in the symmetric case for the three methods when the inverse filter length equals 150. TABLE 5: Crosstalk cancellation performance in the asymmetric case for the three methods when the inverse filter length equals 150.
TABLE 4: Mean crosstalk cancellation performance in the symmetric case for the three methods when the inverse filter length equals 150. TABLE 5: Crosstalk cancellation performance in the asymmetric case for the three methods when the inverse filter length equals 150.
Figure 5: Mean signal-to-distortion ratio (SDR) at different inverse filter lengths for the three methods: the least-squares method (LS), the single-filter structure method (SF), and the CAPZ method (white noise added to HRTF).
Figure 5: Mean signal-to-distortion ratio (SDR) at different inverse filter lengths for the three methods: the least-squares method (LS), the single-filter structure method (SF), and the CAPZ method (white noise added to HRTF).
Figure 6: Mean signal-to-crosstalk ratio (SCR) at different inverse filter lengths for the three methods: the least-squares method (LS), the single-filter structure method (SF), and the CAPZ method (white noise added to HRTF).
Figure 6: Mean signal-to-crosstalk ratio (SCR) at different inverse filter lengths for the three methods: the least-squares method (LS), the single-filter structure method (SF), and the CAPZ method (white noise added to HRTF).
Ficure 7: Impulse responses of crosstalk cancellation in the asymmetric case.
Ficure 7: Impulse responses of crosstalk cancellation in the asymmetric case.
Ficure 1: Block diagram of the proposed 3D binaural synthesis method.
Ficure 1: Block diagram of the proposed 3D binaural synthesis method.
FiGurE 2: Tetrahedral microphone array for 3D DOA estimation.
FiGurE 2: Tetrahedral microphone array for 3D DOA estimation.
3.3. Array Geometry and DOA Estimation. Now consider a tetrahedral microphone array (M = 4) with base radius R, as shown in Figure 2. The sensor location vectors in the 3- dimensional space with origin in the array base center, are given by The DOA vector of the nth source as a function of the azimuth 6, and elevation ¢,, angles is defined as
3.3. Array Geometry and DOA Estimation. Now consider a tetrahedral microphone array (M = 4) with base radius R, as shown in Figure 2. The sensor location vectors in the 3- dimensional space with origin in the array base center, are given by The DOA vector of the nth source as a function of the azimuth 6, and elevation ¢,, angles is defined as
Ficure 3: DOA analysis of a mixture of 4 speech sources. (a) Source locations in the 3D space. (b) Distribution of DOA estimates in the XY plane. (c) Distribution of DOA estimates in the ZY plane.
Ficure 3: DOA analysis of a mixture of 4 speech sources. (a) Source locations in the 3D space. (b) Distribution of DOA estimates in the XY plane. (c) Distribution of DOA estimates in the ZY plane.
Figure 4: Waveforms of real and sparsity-based synthesized binaural signals for different number of sources. (a) One source. (b) Two sources. (c) Three sources.
Figure 4: Waveforms of real and sparsity-based synthesized binaural signals for different number of sources. (a) One source. (b) Two sources. (c) Three sources.
Ficure 5: Simulation setup used to obtain the signals used in the subjective evaluation.
Ficure 5: Simulation setup used to obtain the signals used in the subjective evaluation.
Figure 6: Results of the subjective tests using synthetic mixtures. Black dots denote the mean values and thin bars represent 95% confidence intervals. (a) Sound quality evaluation. (b) Spatial impression evaluation.
Figure 6: Results of the subjective tests using synthetic mixtures. Black dots denote the mean values and thin bars represent 95% confidence intervals. (a) Sound quality evaluation. (b) Spatial impression evaluation.
Ficure 7: Tetrahedral array and acoustic dummy-head used in the experiments.
Ficure 7: Tetrahedral array and acoustic dummy-head used in the experiments.
Figure 8: Results of the subjective tests using real recorded mixtures. Black dots denote the mean values and thin bars represent 95% confidence intervals. (a) Sound quality evaluation. (b) Spatial impression evaluation.
Figure 8: Results of the subjective tests using real recorded mixtures. Black dots denote the mean values and thin bars represent 95% confidence intervals. (a) Sound quality evaluation. (b) Spatial impression evaluation.
Figure 1: General structure of the PIC detector for piano chords.
Figure 1: General structure of the PIC detector for piano chords.
Ficure 2: General structure of the PIC detector.
Ficure 2: General structure of the PIC detector.
Figure 3: Stage m of the PIC detector for note I.
Figure 3: Stage m of the PIC detector for note I.
Ficurs 4: Structure of the Note Decision stage.
Ficurs 4: Structure of the Note Decision stage.
where each column represents all the thresholds found for a dominant group, j. Construction of the Matrix of Thresholds. All the thresholds defined are stored in a matrix with the following structure:
where each column represents all the thresholds found for a dominant group, j. Construction of the Matrix of Thresholds. All the thresholds defined are stored in a matrix with the following structure:
Ficure 5: Structure of the harmonic tests.
Ficure 5: Structure of the harmonic tests.
Figure 6: Block diagram of the octave/fifth test.
Figure 6: Block diagram of the octave/fifth test.
Ficure 7: Comparison of note error rates for different sets of can- cellation parameters and different number of parallel interference cancellation stages.
Ficure 7: Comparison of note error rates for different sets of can- cellation parameters and different number of parallel interference cancellation stages.
Ficure 8: Comparison of note error rates for different polyphony numbers using the proposed PIC detector and the selected reference method proposed in [4]. Polyphony number known in both methods.
Ficure 8: Comparison of note error rates for different polyphony numbers using the proposed PIC detector and the selected reference method proposed in [4]. Polyphony number known in both methods.
Ficure 9: Note error rates for octaves, perfect fifths and other intervals using the proposed PIC detector when the polyphony number is 2.
Ficure 9: Note error rates for octaves, perfect fifths and other intervals using the proposed PIC detector when the polyphony number is 2.
Figure 10: Note error rates for different polyphony numbers using the proposed PIC detector. Polyphony number estimated.
Figure 10: Note error rates for different polyphony numbers using the proposed PIC detector. Polyphony number estimated.
Ficure 11: Note error rates in different levels of noise for different polyphony numbers using the proposed PIC detector. Polyphony number estimated.
Ficure 11: Note error rates in different levels of noise for different polyphony numbers using the proposed PIC detector. Polyphony number estimated.
2.2.2. Acoustic Timbre Features. Acoustic timbre features consist of spectral shape features and spectral contrast features, which are known to be effective in detecting musical moods [8, 9]. The spectral shape features are represented by spectral centroid S3(t), spectral width S4(t), spectral rolloff S5(t), and spectral flux S¢(t):
2.2.2. Acoustic Timbre Features. Acoustic timbre features consist of spectral shape features and spectral contrast features, which are known to be effective in detecting musical moods [8, 9]. The spectral shape features are represented by spectral centroid S3(t), spectral width S4(t), spectral rolloff S5(t), and spectral flux S¢(t):
*7-band octave filter bank. TaBLE 1: Acoustic features representing musical mood.
*7-band octave filter bank. TaBLE 1: Acoustic features representing musical mood.
Figure 1: Overview of QBE retrieval system based on genre classification shift. Controlling the volume balance causes a genre classification shift of a query song, and our system returns songs that are similar to the genre-shifted query.
Figure 1: Overview of QBE retrieval system based on genre classification shift. Controlling the volume balance causes a genre classification shift of a query song, and our system returns songs that are similar to the genre-shifted query.
Frcure 2: Distributions of the first and second principal components of extracted features from the no. 1 piece of the RWC Music Database: Popular Music. Five figures show the shift of feature distribution by changing the volume of the drum part. The shift of feature distribution causes the genre classification shift.
Frcure 2: Distributions of the first and second principal components of extracted features from the no. 1 piece of the RWC Music Database: Popular Music. Five figures show the shift of feature distribution by changing the volume of the drum part. The shift of feature distribution causes the genre classification shift.
where f is a parameter for extracting stable peak and valley values, which is set to 0.2 in our experiments. Here, the spectral contrast features are represented by spectral peak S;(i,t), spectral valley Sg(i,t), and spectral contrast S9(i, t):
where f is a parameter for extracting stable peak and valley values, which is set to 0.2 in our experiments. Here, the spectral contrast features are represented by spectral peak S;(i,t), spectral valley Sg(i,t), and spectral contrast S9(i, t):
Figure 3: Sorted vector of power spectrogram.
Figure 3: Sorted vector of power spectrogram.
The definitions of these models are as follows: All parameters of J,i(t, f) are listed in Table 2. Here, My and Np are the numbers of Gaussian kernels that represent tem- poral and frequency structures of the harmonic tone model, respectively, and M; and N; are the numbers of Gaussians that represent those of the inharmonic tone model. 6 and x are coefficients that determine the arrangement of Gaussian kernels for the frequency structure of the inharmonic model. If 1/(log 8) and x are set to 1127 and 700, F (f) is equivalent to the mel scale of f Hz. Moreover tes Bee ; uv). and vi satisfy the following conditions:
The definitions of these models are as follows: All parameters of J,i(t, f) are listed in Table 2. Here, My and Np are the numbers of Gaussian kernels that represent tem- poral and frequency structures of the harmonic tone model, respectively, and M; and N; are the numbers of Gaussians that represent those of the inharmonic tone model. 6 and x are coefficients that determine the arrangement of Gaussian kernels for the frequency structure of the inharmonic model. If 1/(log 8) and x are set to 1127 and 700, F (f) is equivalent to the mel scale of f Hz. Moreover tes Bee ; uv). and vi satisfy the following conditions:
FIGURE 4: Overall, temporal, and frequency structures of the harmonic tone model. This model consists of a two-dimensional Gaussiar Mixture Model, and it is factorized into a pair of one-dimensional GMMs.
FIGURE 4: Overall, temporal, and frequency structures of the harmonic tone model. This model consists of a two-dimensional Gaussiar Mixture Model, and it is factorized into a pair of one-dimensional GMMs.
TABLE 2: Parameters of integrated tone model. With these functions, the separated power spectrograms, Xu? (t, f) and X{??_,, (t, f), are obtained as
TABLE 2: Parameters of integrated tone model. With these functions, the separated power spectrograms, Xu? (t, f) and X{??_,, (t, f), are obtained as
The spectrogram distribution functions are calculated by minimizing Q“) for the functions. Since the functions satisfy the constraint given by (18), we use the method of Lagrange multiplier. Since Q is a convex function for the spectrogram distribution functions, we first solve the simulteneous equations, that is, derivatives of the sum of Q) and Lagrange multipliers for condition (18) are equal to zero, and then obtain the spectrogram distribution functions, To evaluate the effectiveness of this separation, we use an objective function defined as the Kullback-Leibler (KL) divergence from x, f) and beams f) to each Gaussian kernel of the harmonic and inharmonic models:
The spectrogram distribution functions are calculated by minimizing Q“) for the functions. Since the functions satisfy the constraint given by (18), we use the method of Lagrange multiplier. Since Q is a convex function for the spectrogram distribution functions, we first solve the simulteneous equations, that is, derivatives of the sum of Q) and Lagrange multipliers for condition (18) are equal to zero, and then obtain the spectrogram distribution functions, To evaluate the effectiveness of this separation, we use an objective function defined as the Kullback-Leibler (KL) divergence from x, f) and beams f) to each Gaussian kernel of the harmonic and inharmonic models:
TABLE 3: Number of musical pieces for each genre.
TABLE 3: Number of musical pieces for each genre.
(a) genre classification shift caused by changing the volume of vocal. Genre with the highest similarity changed from rock to popular and to jazz Ficure 6: Ratio of average EMD per genre to average EMD of all genres while reducing or boosting the volume of single instrument part. Here, (a), (b), and (c) are for the vocal, guitar, and drums, respectively. Note that a smaller ratio of the EMD plotted in the lower area of the graph indicates higher similarity. (a) Genre classification shift caused by changing the volume of vocal. Genre with the highest similarity changed from rock to popular and to jazz. (b) Genre classification shift caused by changing the volume of guitar. Genre with the highest similarity changed from rock to popular. (c) Genre classification shift caused by changing the volume of drums. Genre with the highest similarity changed from popular to rock and to dance.
(a) genre classification shift caused by changing the volume of vocal. Genre with the highest similarity changed from rock to popular and to jazz Ficure 6: Ratio of average EMD per genre to average EMD of all genres while reducing or boosting the volume of single instrument part. Here, (a), (b), and (c) are for the vocal, guitar, and drums, respectively. Note that a smaller ratio of the EMD plotted in the lower area of the graph indicates higher similarity. (a) Genre classification shift caused by changing the volume of vocal. Genre with the highest similarity changed from rock to popular and to jazz. (b) Genre classification shift caused by changing the volume of guitar. Genre with the highest similarity changed from rock to popular. (c) Genre classification shift caused by changing the volume of drums. Genre with the highest similarity changed from popular to rock and to dance.
Ficure 7: Genres that have the smallest EMD (the highest similarity) while reducing or boosting the volume of two instrument parts. (a), (b), and (c) are the cases of the vocal-guitar, vocal-drums, and guitar-drums, respectively. (a) Genre classification shift caused by changing the volume of vocal and guitar. (b) Genre classification shift caused by changing the volume of vocal and drums. (c) Genre classification shift caused by changing the volume of guitar and drums.
Ficure 7: Genres that have the smallest EMD (the highest similarity) while reducing or boosting the volume of two instrument parts. (a), (b), and (c) are the cases of the vocal-guitar, vocal-drums, and guitar-drums, respectively. (a) Genre classification shift caused by changing the volume of vocal and guitar. (b) Genre classification shift caused by changing the volume of vocal and drums. (c) Genre classification shift caused by changing the volume of guitar and drums.
Figure 8: Normalized EMDs that are shifted to 0 when the volume control ratio is 0dB for each genre while reducing or boosting thi volume. (a), (b), and (c) graphs are obtained by changing the volume of the vocal, guitar, and drum parts, respectively. Note that a smalle EMD plotted in the lower area of each graph indicates higher similarity than the one without volume controlling. (a) Normalized EMDs by changing the volume of vocal. (b) Normalized EMDs by changing the volume of guitar. (c) Normalized EMDs by changing the volume o drums.
Figure 8: Normalized EMDs that are shifted to 0 when the volume control ratio is 0dB for each genre while reducing or boosting thi volume. (a), (b), and (c) graphs are obtained by changing the volume of the vocal, guitar, and drum parts, respectively. Note that a smalle EMD plotted in the lower area of each graph indicates higher similarity than the one without volume controlling. (a) Normalized EMDs by changing the volume of vocal. (b) Normalized EMDs by changing the volume of guitar. (c) Normalized EMDs by changing the volume o drums.