Searching for  a global synthesis technique

through a quantum conception of sound.

ABSTRACT:   Synthesis techniques based over a discreet conception of sound are
capable of creating sounds well deployed in time and frequency. Yet, there are two
main   groups   of   granular   techniques   that   seem   to   be   antagonistic.   The   analysis­
resynthesis techniques, which favor the creation of sounds where we have an accurate
control  over their  spectral  micro­structure,  and the  traditional  granular  techniques,
which   favor   the   creation   of   sounds   on   a   macro­scale   basis.   This   paper   develops
different ideas for possible connections between these two apparently opposed realms,
that would then aloud the creation of a new global synthesis technique, capable of
producing sounds both at the micro and at the macro time level, while favoring at the
same time their spectral and morphological qualities.

Analysis­resynthesis granulation techniques and traditional granular synthesis
techniques1  (QSGS,  AGS,   Granular   Sampling2),   have   the   same   discrete   sound
conception: they both use elementary  particles  (or "grains") 3  for the generation of
sound signals. Nevertheless, these two large groups of techniques have different ways
of constructing or reconstructing the sound signal, and each one of them contemplates
a   different   way   of   sound   synthesis.   Analysis­resynthesis   granulation   is   based   on
having an accurate control over the partials of a spectral analysis of sound (frequency
domain),  while traditional  granular techniques  are centered  on having free control
over elementary particles in time without contemplating a precise control over the
sound spectrum (time­domain). The mistake of the first group of techniques is that it
doesn’t consider or has the faculty of controlling sound particles with grate flexibility
over the time domain, while the error of the second group is to be incapable of having
a precise control over the spectral sound domain.

One of the main goals of this paper is to establish connections between the
time and the frequency domains in sound synthesis through the granular techniques,
because   they   have   an   essentially   discrete   character   and   allow   us   to   consider   the
creation   of   sounds   well   displayed   in   both   fields.   In  spite   of   this,   it   is   sometimes
necessary to use certain granular techniques when we want to work accurately over
the frequency domain and other granular techniques when we want to work over the
fields of texture, mass and rhythm. But, are there possible connections between the
different granular techniques, or could we choose one same technique and use it for
the generation of sounds both on the micro and on the macro time domains, or for
1 I will call “traditional granular synthesis techniques" the granular
techniques with a non-analytical orientation, with exception of the
formantic granular techniques (CHANT, VOSIM), and the synchronous
granular techniques.
2 The terms Quasi-Synchronous Granular Synthesis (QSGS) and
Asynchronous Granular Synthesis (AGS) have been created by Cuartis
Roads (Roads, 1991). The term Granular Sampling has been proposed by
Cort Lippe (Lippe, 1993).
3 Also known as sound quanta (Gabor, 1946).
creating spectral sounds, as well as sounds whose partials are displayed freely in the
time­frequency   space?   The   traditional   granular   synthesis   techniques   would   not   be
capable of this task because they have a non­analytical orientation (Vaggione, 1993).
Therefore, would the analysis­resynthesis techniques be capable for this job?

Research on the fields of granulation by analysis­resynthesis has been based
on the timbre aspect of sound in the micro­time domain, and the algorithms created
for the transformation of sound through the modification of the analysis has always
been limited by the complications that show­up when we reconstruct the sound signal.
These two aspects I think, are the principal limitations of this group of techniques.
Working on the macro­time domain in this field has been limited to temporal sound
stretching, or to the manipulation of long sounds (where the quantity of RAM memory
needed   becomes   too   expensive).   In   addition,   in   the   case   of   sound   expansion,   the
temporal   transformation   of   sounds   does   not   produce   too   interesting   variations   of
timbre4. 

Spectral like synthesis techniques unfolded in the macro­time domain are more
effective by way of other non­analytical techniques such as additive synthesis. The
variation of spectral partials in this type of synthesis is very flexible thanks to the
software developed for a precise control of the evolution of each partial. Of course,
we   are   speaking   here   of   a   non­analytical   technique,   and   the   analysis­synthesis
techniques are here an essential complement for it. However, what do we do if we
want to obtain sound effects with a morphological character, or which are to be placed
between   the   spectral   continuity   of   waves   and   the   discontinuous   domain   of   sound
particles? Unfortunately,  neither  the additive  synthesis  nor the analysis­resynthesis
techniques have permitted up to the present time to attain good results over these two
territories.

Through my research about the granular synthesis techniques (Rocha Iturbide,
1999), one of my propositions is to use traditional granular synthesis in a similar way
to additive synthesis (Figure 1). This would aloud us to have spectral sounds that can
evolve towards textured sounds (Figure 2), and even towards a total discontinuity
which then becomes rhythm. However, in the spectral domain we will always have
modulation   effects   that   we   cant   control.   With   the   analysis­resynthesis   granular
techniques   we   could   avoid   this   type   of   effects,   but   we   cannot   treat   the   analysis
windows as if they were grains that move freely, because we would then loose control
over the phases, and we would have undesirable sound effects as a result. However,
the idea of having a random control over the analysis windows has already existed and
been used under certain conditions where the noise effect is desired. 

Xavier   Serra   and   Julius   Smith   (1990)   have   proposed   an   analysis­synthesis

technique   by   spectral   modeling   that   uses   a   combination   of   deterministic   plus

4 On the contrary, with granular sampling we can obtain very interesting

timbre transformations.
 stochastic decomposition. They use the deterministic part for the Fourier like partials
that evolve through time, and the stochastic decomposition for the recreation of the
noise elements which are present in the attack portion or during the production of a
sound (for example the noise produced by an arc, by the breath of the voice, etc) and
which   cannot   be   analyzed   in   a   deterministic   way   with   the   Fourier   transform
(Vaggione, 1993). With this procedure, Serra and Smith have confronted the acoustic
dualism (at the level of micro­time) between the continuous control by the analysis of
grains and the discontinuous control of the grains, having obtained good results. So,
why shouldn't we develop this idea and apply it to the domain of micro­time?

a) b)

Fréquence Fréquence

Temps Temps

Figure   1.   ­   a)   traditional   additive   synthesis   b)   granular   additive   synthesis.   The

discontinuities, the overlapping of grains and a possible utilization of a BW of 10 Hz
for every spectral band, make the spectrum of b) different from that of   a). (Rocha
Iturbide, 1999.

Fréquence

Temps

Figure   2.   ­   Evolution   between   a   granular   harmonic   cloud   of   additive   type   and   a

granular textural cloud (Rocha Iturbide, 1999).

The   transitions   between   spectral   sounds   and   sound   disturbances   (Xenakis,

1971) would be possible by using analysis­resynthesis techniques. We would only
need to have different types of controls over the analysis windows or "elementary
grains": one which corresponds to the typical algorithms with which we master the
frequency variations in time (without phase disturbances), and the other, completely
free, with which we can make evolve each grain in a totally independent way through
stochastic, chaotic, and other type of algorithms.

We could imagine a short time Fourier analysis with a window of 1024 points.
If we have a sample rate of 44.1 kHz, our window size will be equivalent to a "grain"
of 23.2 milliseconds. If we produce a non­linear stretching, the time­dilation values
will start to change in time, and we will start to separate the windows having always a
rate that varieties in a constant way. If we suddenly apply an algorithm that separates
the windows in a discontinuous way, and we start to change also the frequency of
each   window,   we   will   have   sound   artifacts,   but   then,   the   windows   will   become
autonomous grains and we will have an asynchronous granular synthesis of sound.
Concerning the region where there is a phase disorder, I think that we may be able to
foresee   in   a   statistic   way  the   type   of   noise   that   we  will   obtain   in   relation   to   the
algorithm that we want to use.

Serra and Smith (Serra & Smith, 1990) have used a statistic control over the
analysis windows by using always the same window size. However, we could also
vary the size of the windows at the moment where we start to separate them in a
discontinuous way; enlarging them and obtaining "grains" which become every time
bigger, and which could become sound textures and rhythmic motives. The  Malvar
wavelets  (Meyer,   1992)   for   example,  would   be   ideal   for   this   kind   of   processes,
because they are grains with musical characteristics (their attack, body and decay are
similar to instrumental short sounds), and also, their size changes through time, as
well as their envelope5.

The process I just described is not the only one possible. The idea of applying
stochastic algorithms  to analysis "elementary grains" could be realized in different
ways. For example, with the wavelets, we have "grain" scales of different length (that
is, different layers of grains, each layer having grains of different length). We could
deconstruct a sound signal in a granular way before generating the resynthesis, by
eliminating in a statistic way grains from different scales. This granulation could be
produced in a progressive way between the different scales, and we could achieve in
this way filtering effects by way of spectral zones; only, here the filtering will be
discrete and discontinuous, because we will always leave some grains at the different
regions which we have filtered6. 

5 The Malvar Wavelets would be well adapted for the analysis of sound
signals. Meyer, one of the inventors of these grains, has suggested. “The
possibility of cutting a signal in a uniform way will help particularly the
researchers that analyze music and speech” (Meyer, 1992).
6 The essence of this idea has been taken from the algorithm degranular

synthesis belonging to QuickMQ software (Steve Berkley, 1995). The main

difference is that the degranulation with this algorithm always gives very
 This  "strain" process has to do with sound transformations in the micro­time
domain, but we could also create different time expansions and contractions for every
grain   layer,   at   the   same   time   of   creating   the   granular   deconstruction,   or   on   the
contrary,   we   could   combine   the   "watery"   time   changing   effect   (discovered   by
Kronland   Martinet,   1991)   with   a   super­production   of   wavelets   controlled   in   a
stochastic way, by using different algorithms and different densities over each grain
layer.

We can imagine other ways of controlling elementary analysis "grains". For
example,   the   idea   of   having   different   analysis   techniques   combined   in   the   same
algorithm (Best Basis), which has been proposed by Coifman (1992), could be used in
music for the analysis, the transformation and the resynthesis of sound. This way, we
would have a palette of different grains at the moment of the signal reconstruction.
Therefore,  we would have an enormous  variety  of possibilities  for the mixture  of
current analysis­resynthesis transformations and traditional granular transformations.
Yet, could this mixture really lead us towards the creation of a global synthesis, with
 
which we would be able to work simultaneously in the different time scales? 7.

We know well the problems we may encounter with the analysis of a sound
signal, particularly those which concern Heisenberg's principle of incertitude (Gabor,
1946; Wiener, 1964). On the other hand, we know that the global analysis of a signal
displayed   both   over   micro   and   macro   time,   will   always   be   difficult,   because   the
interaction between the different time scales has a non­linear character 8, and besides,
we   need   to   consider   also   the   significance   of   musical   context   (Vaggione,   1993).
Nevertheless, a series of new techniques (wavelets) and their hybridization could be of
precious aide for the amelioration of this situation. The principle of incertitude will
always   be   there,   but   "elementary   grains"   such   as  wavelets,  that   function   in   the
different   time   scales   (and   which   are   precise   in   time),   in   combination   with   more
precise   grains   in   the   domain   of   frequency,   could   be   combined   to   obtain   a   more
accurate global analysis9. 

similar results, while I propose a more accurate and subtle filtering, which
wavelets can aloud thanks to their time-scale characteristic.
7 This idea by itself constitutes a quantum paradigm. The analysis-

synthesis techniques take advantage of the wave property of an

“elementary grain” while traditional granular techniques take advantage of
the particle property. Yet, in reality, the "elementary grain" has a constant
dual behavior; so there has to be a synthesis technique that takes
advantage of these two aspects at the same level.
8 However, for certain musical sounds, we can realize an effective global

analysis of the typical features of time domain. In accordance with

Eldénius (1995), we can find different affinities between the different small
metric cells and the large metric structures in much of the musical
literature, like for example in the Indian ragas or in the golden section of
Bartok's music.
9 “Because of the wave/particle duality, the constituent members of particle

systems carry at all times the properties of both waves and particles. With
 In that which concerns the transformation of the resynthesis of a sound signal,
we have the same problems we faced with the analysis, but our attitude is much more
relaxed and flexible, because here we are interested in the creation of new sounds by
keeping a complete control at the moment of producing them, but being able to have
certain margin of error that could be predictable with a bit of experience. Non­linear
interaction   between   the   different   time   scales   complicates   the   task   of   producing   a
global synthesis that is unfolded simultaneously through micro and macro time. For
Vaggione,  we  cannot  have  a  common  syntax  while  working  in the  different  time
domains (Vaggione, 1993). 

It is true that if we want to unite analysis­resynthesis techniques (that work
better in the micro­time domain), with traditional granular synthesis techniques (that
work  better   in   the   macro­time   domain)   we  have   to   resort   to   a   series   of   different
control algorithms that have different syntax conceptions. This gives possible way to
the   creation   of   a   synthesis   technique,   which   gathers   the   domains   of   time   and
frequency in a balanced way. However, we will always have non­controllable effects,
yet   which  we  can   more  or  less  foresee  and   use   to  our  advantage.  May  be,  these
inevitable undetermined moments ­where we are far away from the accurate control of
frequencies and from the stochastic control of texture, mass and rhythm­ could be
observed, accepted, and valued in a musical way with a cageian philosophy10. 

Two last remarks: we are not obliged to have a stochastic control over the
grains   while   working   in   the   macro­time   domain;   we   could   have   for   example,
prototype   granular   structures   preconceived   and   which   are   triggered   by   standard
algorithms which could also be transformed through time (Figure 3) (Rocha Iturbide,
1999)11. On the other hand, my propositions in this paper will remain theoretic (and
may   be   a   utopia?)   because   unfortunately   I   am   not   a   mathematician,   nor   a   signal
processing   specialist,   and   consequently,   I   do   not   have   the   possibility   of   applying
them. Yet, I hope that they will have some utility, and that other researches will be
interested  in using them  and experiment  with them.  However, the  idea of joining
micro­time with macro­time in one synthesis technique has been developed in theory
and   in   practice   by   me   and   Gerhard   Eckel   (Eckel,   Rocha   Iturbide,   1995;   Rocha
Iturbide, 1999) by means of a formantic granular synthesis technique (GiST) that uses

their particle aspect, they have the capacity to be something in particular

which can be ‘pinned down’, if only briefly and only somewhat. With their
wave aspect they have the capacity to relate to other ‘individuals’ through
the partial overlapping of their wave functions” (Zohar, 1990. pg 95).
10 The term Cagean has been taken from composer John Cage, who worked

all his life with the idea of indetermination.

11 This only makes sense when we have various grains per second and

with a long enough duration as to be considered as small notes. At this

moment, the grains create rhythmic motives, and it is then logical to
structure them in a determinist way. This subject has been developed in the
sixth chapter of my doctoral thesis: "Les techniques granulares dans la
synthèse sonore" (Rocha Iturbide, 1999).
 FOF's (Rodet, 1979) as fundamental grains. This technique (which doesn’t belong to
the   analysis­synthesis   techniques)   works   well   in   the   time   and   in   the   frequency
domains,   and   it   could   may   be   incorporated   to   the   analysis­resynthesis   techniques
(Rodet et d’Alessandro, 1989) and serve as “lapis philosophorum” for the creation of
a multi­scale sound synthesis technique. 

Fréquence

Temps

Figure 3.­ Interaction between different micro and macro granular processes on the 
same temporal level. The globalization of all these processes would constitute a 
musical composition (Rocha Iturbide, 1999).

References

­Berkeley, S.W. 1995. “QUICKMQ: A software tool for the modification of time­varying spectrum 
analysis files”. ICMC proceedings, Banff Canada: ICMA.
­Coifman, R.R., Wickerhauser, M.V. 1992. “Entropy Based Algorithmes for Best Basis Selection”. 
IEEE Transactions on Information Theory, vol. 32, mars 1992.
­Eckel, G., Rocha Iturbide, M. 1995. “The develeppment of GiST, a Granular Synthesis Toolkit 
Based on an Extention of the FOF Generator”.  In Proceedings of the 1995 international computer 
conference, Banff Canada: ICMA.
­Eldenius, M. 1995. “Fractal Structures and Formalized Composition”. In Proceedings of the 1995 
international computer conference, Banff Canada: ICMA.
­Gabor, Dennis. 1946. “Theory of communication.” Journal of the institute of Electrical Enginner Part
III, 93.
­Kronland­Martinet, R.  & Grossman A. 1991 “Application of time frequency and time­scale 
methods (wavelet transforms) to the analysis, synthesis and transformation of natural sounds”. In GG. 
de Poli, A. Piccialli & C. Roads, editors, The representation of musical signals. MIT Press. 87­118.
­Lippe, C. 1993. “A Musical Application of Real­time Granular Sampling Using the IRCAM Signal 
Processing Workstation”. In: T. Taguiti. ed. Proceedings of the 1994 International Computer Music 
Conference : ICMA.
­Meyer, Y. 1992. “Les Ondelettes. Algorithmes et Applications”. Armand Colin, Paris.
­Roads Curtis. 1991. “Asynchronous granular synthesis”. Dans G.De Poli, A. Piccialli, and C.Roads, 
eds. Representations of musical signals. Cambridge, Massachusets: The MIT Press.
­Rocha Iturbide, M. 1999. "Les techniques granulaires dans la synthèse sonore". Doctoral thesis under
the direction of Horacio Vaggione. University of Paris VIII. France.
­Rodet, X. 1979/1984. “Time­Domain Formant­Wave­Function Synthesis”. Proceedings of the NATO­
ASI . Meeting Bonas, rééd. Computer Music Journal 8(3) : 9­14.
­Rodet, X. et d’Alessandro C. 1989. “Synthèse et analyse­synthèse par fonctions d’ondes 
formantiques”. Dans Journal Acoustique 2 163­169.
­Serra, X.  "A system for Sound Analysis /Transformation/ Synthesis   based on a deterministic plus
stochastic decemposition". PhD thesis, CCRMA, Stanford University. 
­Serra,   X.,   Smith,   J.O.  1990.   “Spectral   Modeling   Synthesis:   A   sound   Analysis/Synthesis   System
based on a Deterministic plus Stochastic Decomposition”. Computer Music Journal, Vol 16 (4).
­Vaggione,   H.  1993.  “Timbre   as   syntax:   a   spectral   modeling   approach”.  Science   and   Music
Conference, City University of London.
­Wiener, N. 1964. “Spatial­Temporal Continuity, Quantum Theory and Music”. Dans M. Capek (ED.) 
1975: The Concepts of Space and Time. Boston, Reidel.
­Xenakis I. 1971. “Formalized music”. Bloomington: Indiana University Press.
­Zohar, Danah.1990.  The Quantum self. Bloomsbury Publishing Ltd. UK.