Speech to Song Illusion: Evidence from Mandarin Chinese

future experiment to probe into the robustness oftone pitch intervals in real-time speech (by usingcorpus methods) and its effect on intelligibility.Furthermore, I demonstrate how the intervalstructure (1.b) plays a less significant role whenthe duration condition is satisfied in (1.a) in theperception of speech to song illusion. All speechanalysis, synthesis, and manipulation are done onPraat (www.praat.com).II Analysis of Deutsch’s DataTo analyze Deutsch’s data, I pay particularattention to two data sets. In Deutsch’sexperiment, it was interesting to compare the twosets of recordings, one by the group (a) of 11subjects who were asked to reproduce the phraseafter having heard it for ten times, the other bygroup (b) of 11 subjects who only heard it oncebut were asked to reproduce the phrase as sungphrase. Their output exhibited striking similarityas is shown in the graph below in average pitch.Figure 2This difference suggests that by focusing on theacoustic properties (or imitating the prosody) ofthe speech sentence one might produce the sungphrase (sung phrase is usually produced byexaggerating the acoustic features of thesentence, such as prolonged pitch stability). Thefollowing graphs show my calculation of themean speed of pitch change at a 0.01s intervalfor the two sentences from above, one fromgroup (a) and the other from group (c).Table 1Figure 1Meanwhile, there is striking difference betweenthe output of group (a) and group(c) who onlyheard the phrase once and were simply asked torepeat:Table 2It is evident from the data in the graph 1 and 2that the speed of pitch change in sung-like2

In the following section I consider the twoparttonal hypothesis proved to be crucial to thegeneration of speech to song illusion in theexperiments on German by Falk and Rathecke byusing preliminary evidence from experiments ongenerating speech to song illusion in MandarinChinese. Here I focus on the first tone and thirdtone of MC. It is known that in connectedspeech, tone 1 (high level) and tone 3(lowfalling, also known as half-third-tone due to itsinconsistency with the citation form in mostcases in natural speech), unlike tone 2(midrising) and tone 4(high falling), are more likelyto have stable pitches therefore facilitating thegeneration of the illusion. (or rather because thetone 2 and tone 4 have relatively large intervalsin their contours, they are most unlikely to besung-like in real speech).III Target Stabilitystatic tone or vowel. An illustration of thisscenario is shown in Figure 3(a).2. A target is given just enough time, e.g.,when a monophthong vowel is 75 ms long, or astatic tone is 150 ms long—Within the allottedtime interval, the movement toward the targetwould proceed continuously, and the target valuewould not be achieved until the end of itsallotted time interval. An illustration of thisscenario is shown in Figure 3(b).3. A target is given insufficient amount oftime, e.g., a vowel is much less than 75 ms long,or a static tone is much less than 150 ms long—Within the allotted time interval, again themovement toward the target would proceedcontinuously; but the target would not beapproached even by the end of its allotted timeinterval. By the time its cycle ends, however, theimplementation of the target has to terminate andthe implementation of the next target has to start,as is illustrated in Figure 3(c).Recent experiments (Xu 2002; 2004) on themaximum speech of pitch change in speech flowindicated that unlike previously thought, physicallimitation does play an important role in speechproduction and perception. More specifically, themaximum speed of pitch change is actually morefrequently reached in natural speech, and humanlisteners show great robustness to high speedcomputer synthesized speech while havingserious limitations on understanding massiveundershoot in human speech. Here I apply resultsfrom the Target Approximation model in pitchchange. Directly relevant to the current researchquestion on the duration and stability of the toneproduction, Xu’s Target Approximation modeloutlined three hypothetical scenarios, all ofwhich commonly occur in any language:1. There is plenty of time for a target, e.g.,when a monophthong vowel is well over 100 mslong, or a static tone is well over 200 ms long—The target value would be attained before theend of its allotted time; and a pseudo steady stateat that value might be achieved in the case ofFigure 3Based on this model, we expect a cutoff valueof at least 200ms of the mean steady toneduration for the occurrence of the speech to songillusion. In a trial experiment, a male nativespeaker of Mandarin Chinese was asked to speakout the following sample sentence (consisting oftone 1 and tone 3 only) in three different speech4

SPEECH TO SONG ILLUSION: EVIDENCE FROM MC 5rates, fast, mid, and slow, with a broad focus.The pitch intervals of the tones are either 5 st or7 st (perfect fourth or perfect fifth), thusfacilitating the generation of the effect:Table 3Preliminary results of perceptual trialexperiments showed that this cutoff value isindeed responsible for the generation of theeffect. Only the slow speech rate generated theperceptual shift from speech-like signal to thesong-like signal, while the other two do notexhibit this effect no matter how many times therecording is repeated. Moreover, the illusionseems to disappear when synthesizedacceleration is applied on the slow rate sentence.In sum, when the cutoff value of Xu’sTarget Approximation model is applied, thetarget stability is demonstrated to be crucial forthe generation of speech to song illusion. This isin agreement with Falk and Rathecke’s work onGerman. Future perceptual experiments areneeded to decide the details of the illusion inMC.IV Interval StructureThe question of interval structure in thespeech to song illusion is proved to becomplicated in MC, considering the flexibility oftone production and perception in MC, as well asthe interaction between lexical tone andintonation (Shen 1990; Xu 1994). Whenrealizing transitions from one tone to another inreal-time speech in MC, for instance, from thethird tone to the first tone, because the tones aredefined relatively, there is a range of acceptedintervals (i.e. all the different intervals that mightbe interpreted as accepted third tone to first tonetransition in real speech). The robustness of thisinterval is true especially when we consider thatMC preserves an over 90% of intelligibilityspoken in monotone in a non-noisy background(Xu&Patel&Wang 2010). The change in broadand narrow focus semantically also is reflected inprosody, thus in tone intervals.For the current study I use the same samplesentence from above. First I ask a question: Howis tone interval mapped onto the specific tonerelations, such as a transition from tone 3 to tone1? In a series of trial experiments, the samplesentence (with third tone vs. first tone contrast)was read by male native MC speaker using awide range of intervals from smaller than 1 st to12 st. Results indicate that all intervals producedare accepted as normal speech (presumablyheard in different context). One explanation forthis is that when third tone is contrasted with thefirst tone in MC, there is no possible confusionas the other two tones in the inventory are risingand falling tones with large intervals and greaterspeed of pitch change, with no other level tonesto confuse with. Thus, as long as the third tonetransits into a first tone in an ascending interval,presumably it can be accepted as intelligible andtolerable in normal speech. (While on the otherhand, if a descending interval is heard in thiscase it inflects a tone accent from a differentdialect of MC, while still being intelligible inthis case). To confirm this empirically, I intendto build a corpus collecting material in realspeech and study the interval range of the leveltone contrasts.More strikingly, in one trial of the abovementionedreading where the speaker intends tospeak in a small interval (such as 1 st), we foundthat in a tone pattern of (3-1-1-1-1-1), the firstfour words are actually assigned the about samepitch height, while the last two tones are about 1-2 st higher (Table 4). In a similar trial, anascending pattern is found within the realizationof the five first tone words, thus assigning thefive first tone words different pitch heights in anascending fashion.5

syllable wo he san bei ka feicentralpitch(Hz) 85 85 84 84 88 88tone 3 1 1 1 1 1Table 4 Irregular pitch-tone interval mappingin the sample sentenceThe great flexibility of tone intervals inaccepted normal speech indicates that theconstraint of an interval relation that is similar tothe musical scalar relation is little in thegeneration of speech to song illusion. We may aswell speculate that given the sufficient pitchtarget stability, the occurrence of speech to songillusion depends less on the tone intervalstructure. Moreover, due to the listener’s abilityto adjust perceived pitch in different tuningsystems to the ones we’re familiar with (such asinterpreting as well-tempered pentatonic scalewhen hearing a Sundanese Salandro scale by theWestern listeners), it is likely that the intervalstructure does not play a significant role in theillusion in the case of MC. Future perceptualexperiments is called for in determining the roleof interval structure as well as its complexmapping onto the tone relations.V DiscussionIn conclusion, the current study confirms theTonal Hypothesis by Falk and Rethecke (2010),and the mechanisms in generating speech to songillusion in Mandarin Chinese. In the meantime,the current study does not negate the conclusionby Deutsch. First of all, we do have to take intoaccount part of the physical properties as a preconditionin generating the speech to songillusion; nonetheless, we do acknowledge thefact that the same input of the same physicalreality of the signal can generate different out putwhen repeated through the human perceptualsystem.There exists a long-standing notion of speechto song continuum by ethnomusicologistsanthropologists, who found in many culturesaround the world the existence of the vocalperforming genres that lie between speech andsong (or in some cases known as heightenedspeech). In the meantime, as illustrated in thispaper, what happens to speech perception whenrepeated signal is presented to human listenersdid not particularly attract scholar attentions untilrecently. Therefore the research into speech tosong illusion may offer us a new passage way tonavigate the human audio-perceptual system andbrain modularity from speech to song.VI References[1] Deutsch, D., 1995. Musical Illusions andParadoxes. CD: Philomel Records.[2] Deutsch, D., Lapidis, R., Henthorn, T., 2008. Thespeech-to-song-illusion. J. Acoust. Soc. America,s.124, 2471.[3] Falk, F., Rathcke, T. 2010. “On the Speech-To-Song Illusion: Evidence from German”. SpeechProsody 5 th International Conference, Chicago, IL.[4] Mertens, P. 2009. The Prosogram v.2.7(manual& tutorial).http://bach.arts.kuleuven.be/pmertens/prosogram[5] Patel, A., 2008. Music, language and the brain.Oxford: University Press.[6] Peretz, I., in press. Music, language andmodularity in action. In Rebuschat, P., Rohrmeier,M., Hawkins, J. and Cross, I. [Eds], Language andMusic as cognitive systems. Oxford: University Press.[7] Peretz, I., Champod, S. and Hyde, K. 2003.Varieties of Musical Disorders: The Montreal Batteryof Evaluation of Amusia. Annals of the New YorkAcademy of Sciences 999, 58-75.Peretz, I. Coltheart, M., 2003. Modularity of musicprocessing. Nature neuroscience, 6, 688-691.[8] Pilotti, M., Antrobus, J.S. and Duff, M. 1997.The effect of presemantic acoustic adaptation onsemantic 'satiation'. Memory and Cognition 25 (3),305-312.[9] Sundberg, J., 1987. The Science of the SingingVoice. Illinois: Northern Illinois Press.[10] Sundberg, J., 1989. Synthesis of Singing byrule. In Mathews, M. V. and Pierce, J. R. [Eds],Current Directions in Computer Music Research.6

SPEECH TO SONG ILLUSION: EVIDENCE FROM MC 7MIT Press, 45-55.[11] Sundberg, J. 2006. "The KTH Synthesis ofSinging." in Advances in Cognitive Psychology,2006.v2,no.2-3,pp.131-143.[12] Shen, X.S. 1990. The Prosody of MandarinChinese (University of California Publications inLinguistics). CA: UC Press.[13] Xu, Y. 2004. Understanding tone from theperspective of production and perception. Languageand Linguistics 5: 757-797.[14] Xu, Y. and Sun X. 2002. Maximum speed ofpitch change and how it may relate to speech.Journal of the Acoustical Society of America 111:1399-1413.[15] Xu, Y. 1994. Production and perception ofcoarticulated tones. Journal of the Acoustical Societyof America 95: 2240-2253.[16] Xu, Y. Patel, A. Wang, B. 2010. The role of F0variation in the intelligibility of Mandarin sentences.Speech Prosody 5 th International Conference,Chicago, IL.[17] Zatorre, R.J., Belin, P. and Penhune,V.B., 2002.Structure and function of auditory cortex: music andspeech. Trends in cognitive Science 6, 37-46.7