ECE 398-RS: Introduction to Speech Technology

Richard Sproat

Fall 2006

MW 2:00--3:20, Engineering Hall 106B3

Office Hours: Wednesdays 11:00--12:30, Beckman 2057

NB: This is constantly being revised.

Overview

This new course offers a general introduction to speech technology covering the areas of speech synthesis, speech recognition, speaker transformation, speaker identification and forensic speech applications. The course starts with an overview of human speech production and processing, and then proceeds to a discussion of algorithms common to multiple speech application areas, before delving into the areas listed above. Lectures may be supplemented with student-led discussion of readings from selected areas. The grade will be based upon in-class discussions of readings, regular homeworks, a midterm and final.

Syllabus

Week	Dates	Topic	Reading	Homework
1	8/23	Course intro, Overview of speech technology	Huang et al. Ch. 1	Homework 1
2	8/28, 8/30 (Bob Wickesberg guest lecture)	Phonetics Human audition: Bob Wickesberg's lecture	Stevens, Ch. 1 Ladefoged Chs. 1 - 4 Huang et al. Ch. 2 Gelfand, Chs. 1, 14 Miller & Nicely	Homework 2
3	9/6	Prosody (part of the phonetics slides.)	Ladefoged Ch. 5 (Intonation) Shih, Ch. 2	Homework 3
4	9/11, 9/13	Finite-state machines Estimation/Smoothing and a brief intro to language modeling	Roark &Sproat, Ch. 1 (Finite-state machines) (Roark & Sproat, Ch. 6.1, language modeling and smoothing)
5	9/18, 9/20 (Mark Hasegawa-Johnson guest lecture)	HMMs/Part of speech tagging Acoustic theory of speech production	Huang et al., Ch. 8 (HMMs) Roark & Sproat, 6.2--6.3 Huang et al. 6.2 (Acoustical model of Speech Production)	Homework 4
6	9/25, 9/27	Context-free grammars and parsing Written language	Jurafsky & Martin, Ch. 9 Roark & Sproat, Ch 7
7	10/2, 10/4	Overview of synthesis; Overview of Festival; Pronunciation Modeling;	Jurafsky/Martin forthcoming edition chapter on synthesis. Klatt, 1987 Taylor, Ch 3. Sproat Chapter from Springer book	Homework 5
8	10/9, 10/11	Sense Disambiguation Chilin Shih Guest Lecture: Duration and Intonation	Taylor, Chs. 7, 10	Homework 6
9	10/16, 10/18	Midterm
10	10/23, 10/25	Accent and phrasing prediction (using some slides from here) Guest lecture on LPC (Steve Levinson)	(Taylor, Ch. 7, Prosody prediction) Taylor, Ch. 11-13
11	10/30, 11/1	Synthesis methods	Taylor, Chapters 14-17	You must talk to me by this week about potential projects
12	11/6, 11/8	HMM Synthesis. TTS evaluation	van Santen, 1993
13	11/13, 11/15	Speech Recognition. Language modeling, again.	HTK book, chs. 1-3 Huang et al. ch. 9
G	o	b b	l	e
14	11/27, 11/29	Document structure and markup (Raman's examples in PDF here.) Discussion of some of Nass & Brave's results	Huang et al., Ch. 18 Nass & Brave, Chs 1--3
15	12/4, 12/6	A real application: Speech Data Mining Final (12/6)

Grades

Grades will be based on regular homeworks, a midterm and a final. The exams will account for about 30% of the grade.

Late Homework Policy

I will accept homework up to one week after the due date, but will take off 3% for every day it is late up to a total of 21% of the grade for that homework. (You can submit homework by email or by posting it on a webpage, so you don't have to wait until the next class to turn it in.)

Readings

There is no single text that covers the material of this course so the readings will comprise a selection from a variety of sources. The list below is not comprehensive: we will likely add other readings during the course of the semester. Note that the PDF's are mostly in a password protected directory since much of the material is under copyright. Some of the papers require DjVu, which is available free from Lizardtech.

Gelfand, Stanley. Hearing. Marcel Dekker, 1981. PDF
XD Huang, A. Acero, and H. Hon, Spoken Language Processing, Prentice Hall, 2000. PDF
The HTK Book. http://htk.eng.cam.ac.uk/docs/docs.shtml
Daniel Jurafsky and James H. Martin SPEECH and LANGUAGE PROCESSING: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Prentice-Hall, 2000 PDF
For the upcoming revised edition there is a draft of the chapter on TTS here.
Klatt, D.H., "Software for a Cascade/Parallel Formant Synthesizer", Journal of the Acoustical Society of America, 67.3, 971-995, March 1980. DjVu.
Klatt, D.H., "Review of Text-to-Speech Conversion for English", Journal of the Acoustical Society of America, 82.3, 737-793, Sept 1987. Online version
Ladefoged, Peter. A Course in Phonetics. 5th ed. Heinle & Heinle. 2001. PDF
Miller, George A. and Nicely, Patricia, "An Analysis of Perceptual Confusions among some English Consonants," J.Acoust.Soc.Am. 27:2, 1955 DjVu
Nass, Clifford and Brave, Scott. Wired for Speech. MIT Press. 2005. PDF
Quatieri, Thomas. Discrete-Time Speech Signal Processing. Prentice Hall. 2002. PDF
Brian Roark, Richard Sproat. Computational Approaches to Syntax and Morphology. Oxford, Oxford University Press, Forthcoming 2007. PDF
van Santen, J. P. H., Perceptual experiments for diagnostic testing of text-to-speech systems, Computer Speech and Language, 1993, 7:49--100. PDF.
Shih, Chilin. Prosody Learning and Generation. Submitted to Springer Verlag. 2006. Links.
Sproat, Richard. "Linguistic analysis for speech synthesis." In Jacob Benesty, Yiteng (Arden) Huang and M. Mohan Sondhi (eds.). Springer Handbook on Speech Processing and Speech Communication. Springer. 2006 (forthcoming). PDF
Kenneth N. Stevens, Acoustic Phonetics. Cambridge, Mass.: The MIT Press, 1999. PDF
Taylor, Paul. Text-to-Speech Synthesis. (Draft). 2006. PDF