### Unformatted Attachment Preview

1.1 To compare the earthquake and explosion signals, plot the data displayed in Figure 1.7 on the same graph using dierent colors
or dierent line types and comment on the results. (The R code in Example 1.11 may be of help on how to add lines to existing
plots.)
1.2 Consider a signal-plus-noise model of the general form xt = st + wt , where wt isGaussianwhitenoisewithw2
=1.Simulateandplotn=200observationsfrom each of the following two models.
(a) xt =st +wt,fort =1,...,200,where
⇢
st=
0,
10exp{(t100)}cos(2⇡t/4),
Hint:
s = c(rep(0,100), 10*exp(-(1:100)/20)*cos(2*pi*1:100/4)) x = s + rnorm(200) plot.ts(x)
20
t=1,...,100 t = 101,...,200.
(b) xt = st + wt , for t = 1, . . . , 200, where
⇢
st=
0,
10exp{(t100)}cos(2⇡t/4),
(c) Compare the general appearance of the series (a) and (b) with the earthquake series and the explosion series shown in Figure
1.7. In addition, plot (or sketch) and compare the signal modulators (a) exp{t/20} and (b) exp{t/200}, for t = 1, 2, . . . , 100.
Section 1.2
1.3 (a) Generate n = 100 observations from the autoregression xt = .9xt2 + wt
with w = 1, using the method described in Example 1.10. Next, apply the moving average filter
vt = (xt + xt1 + xt2 + xt3)/4
to xt , the data you generated. Now plot xt as a line and superimpose vt as a dashed line. Comment on the behavior of xt and how
applying the moving average filter changes that behavior. [Hints: Use v = filter(x, rep(1/4, 4), sides = 1) for the filter and note that
the R code in Example 1.11 may be of help on how to add lines to existing plots.]
200
t=1,...,100 t = 101,...,200.
“tsa4_trimmed” — 2017/12/8 — 15:01 — page 39 — #49
.
(b) Repeat (a) but with xt = cos(2⇡t/4).
.
(c) Repeat (b) but with added N(0, 1) noise,
xt =cos(2⇡t/4)+wt. (d) Compare and contrast (a)–(c); i.e., how does the moving average change each
series.
Section 1.3
1.4 Show that the autocovariance function can be written as (s,t)=E[(xs μs)(xt μt)]=E(xsxt)μsμt,
where E[xt] = μt.
1.5 For the two series, xt , in Problem 1.2 (a) and (b):
(a) Compute and plot the mean functions μx (t), for t = 1, . . . , 200.
b) Calculate the autocovariance functions, x (s, t), for s, t = 1, . . . , 200.
1.20 (a) Simulate a series of n = 500 Gaussian white noise observations as in Exam- ple 1.8 and compute the sample ACF, ⇢ˆ(h),
to lag 20. Compare the sample ACF you obtain to the actual ACF, ⇢(h). [Recall Example 1.19.]
(b) Repeat part (a) using only n = 50. How does changing n aect the results?
1.21 (a) Simulate a series of n = 500 moving average observations as in Example 1.9 and compute the sample ACF, ⇢ˆ(h), to lag
20. Compare the sample ACF you obtain to the actual ACF, ⇢(h). [Recall Example 1.20.]
(b) Repeat part (a) using only n = 50. How does changing n aect the results?
“tsa4_trimmed” — 2017/12/8 — 15:01 — page 1 — #1
Springer Texts in Statistics
Robert H. Shumway
David S. Stoffer
Time Series
Analysis and Its
Applications
With R Examples
Fourth Edition
“tsa4_trimmed” — 2017/12/8 — 15:01 — page 2 — #2
Robert H. Shumway
David S. Stoffer
Time Series Analysis and
Its Applications
With R Examples
Fourth Edition
live free or bark
“tsa4_trimmed” — 2017/12/8 — 15:01 — page v — #3
Preface to the Fourth Edition
The fourth edition follows the general layout of the third edition but includes some
modernization of topics as well as the coverage of additional topics. The preface to
the third edition—which follows—still applies, so we concentrate on the differences
between the two editions here. As in the third edition, R code for each example
is given in the text, even if the code is excruciatingly long. Most of the examples
with seemingly endless coding are in the latter chapters. The R package for the text,
astsa, is still supported and details may be found in Appendix R. A number of data
sets have been updated. For example, the global temperature deviation series have
been updated to 2015 and are included in the newest version of the package; the
corresponding examples and problems have been updated accordingly.
Chapter 1 of this edition is similar to the previous edition, but we have included
the definition of trend stationarity and the the concept of prewhitening when using
cross-correlation. The New York Stock Exchange data set, which focused on an old
financial crisis, was replaced with a more current series of the Dow Jones Industrial Average, which focuses on a newer financial crisis. In Chapter 2, we rewrote
some of the regression review, changed the smoothing examples from the mortality
data example to the Southern Oscillation Index and finding El Niño. We also expanded the discussion of lagged regression to Chapter 3 to include the possibility of
autocorrelated errors.
In Chapter 3, we removed normality from definition of ARMA models; while the
assumption is not necessary for the definition, it is essential for inference and prediction. We added a section on regression with ARMA errors and the corresponding
problems; this section was previously in Chapter 5. Some of the examples have been
modified and we added some examples in the seasonal ARMA section.
In Chapter 4, we improved and added some examples. The idea of modulated
series is discussed using the classic star magnitude data set. We moved some of the
filtering section forward for easier access to information when needed. We removed
the reliance on spec.pgram (from the stats package) to mvspec (from the astsa
package) so we can avoid having to spend pages explaining the quirks of spec.pgram,
which tended to take over the narrative. The section on wavelets was removed because
“tsa4_trimmed” — 2017/12/8 — 15:01 — page vi — #4
vi
Preface to the Fourth Edition
there are so many accessible texts available. The spectral representation theorems are
discussed in a little more detail using examples based on simple harmonic processes.
The general layout of Chapter 5 and of Chapter 7 is the same, although we have
revised some of the examples. As previously mentioned, we moved regression with
ARMA errors to Chapter 3.
Chapter 6 sees the biggest change in this edition. We have added a section on
smoothing splines, and a section on hidden Markov models and switching autoregressions. The Bayesian section is completely rewritten and is on linear Gaussian
state space models only. The nonlinear material in the previous edition is removed
because it was old, and the newer material is in Douc, Moulines, and Stoffer (2014).
Many of the examples have been rewritten to make the chapter more accessible. Our
goal was to be able to have a course on state space models based primarily on the
material in Chapter 6.
The Appendices are similar, with some minor changes to Appendix A and Appendix B. We added material to Appendix C, including a discussion of Riemann–
Stieltjes and stochastic integration, a proof of the fact that the spectra of autoregressive
processes are dense in the space of spectral densities, and a proof of the fact that spectra are approximately the eigenvalues of the covariance matrix of a stationary process.
We tweaked, rewrote, improved, or revised some of the exercises, but the overall
ordering and coverage is roughly the same. And, of course, we moved regression with
ARMA errors problems to Chapter 3 and removed the Chapter 4 wavelet problems.
The exercises for Chapter 6 have been updated accordingly to reflect the new and
improved version of the chapter.
Davis, CA
Pittsburgh, PA
September 2016
Robert H. Shumway
David S. Stoffer
“tsa4_trimmed” — 2017/12/8 — 15:01 — page vii — #5
Preface to the Third Edition
The goals of this book are to develop an appreciation for the richness and versatility
of modern time series analysis as a tool for analyzing data, and still maintain a
commitment to theoretical integrity, as exemplified by the seminal works of Brillinger
(1975) and Hannan (1970) and the texts by Brockwell and Davis (1991) and Fuller
(1995). The advent of inexpensive powerful computing has provided both real data
and new software that can take one considerably beyond the fitting of simple time
domain models, such as have been elegantly described in the landmark work of Box
and Jenkins (1970). This book is designed to be useful as a text for courses in time
series on several different levels and as a reference work for practitioners facing the
analysis of time-correlated data in the physical, biological, and social sciences.
We have used earlier versions of the text at both the undergraduate and graduate levels over the past decade. Our experience is that an undergraduate course can
be accessible to students with a background in regression analysis and may include
Section 1.1–Section 1.5, Section 2.1–Section 2.3, the results and numerical parts of
Section 3.1–Section 3.9, and briefly the results and numerical parts of Section 4.1–
Section 4.4. At the advanced undergraduate or master’s level, where the students
have some mathematical statistics background, more detailed coverage of the same
sections, with the inclusion of extra topics from Chapter 5 or Chapter 6 can be used
as a one-semester course. Often, the extra topics are chosen by the students according
to their interests. Finally, a two-semester upper-level graduate course for mathematics, statistics, and engineering graduate students can be crafted by adding selected
theoretical appendices. For the upper-level graduate course, we should mention that
we are striving for a broader but less rigorous level of coverage than that which is
attained by Brockwell and Davis (1991), the classic entry at this level.
The major difference between this third edition of the text and the second edition
is that we provide R code for almost all of the numerical examples. An R package
called astsa is provided for use with the text; see Section R.2 for details. R code
is provided simply to enhance the exposition by making the numerical examples
reproducible.
We have tried, where possible, to keep the problem sets in order so that an
instructor may have an easy time moving from the second edition to the third edition.
“tsa4_trimmed” — 2017/12/8 — 15:01 — page viii — #6
viii
Preface to the Third Edition
However, some of the old problems have been revised and there are some new
problems. Also, some of the data sets have been updated. We added one section
in Chapter 5 on unit roots and enhanced some of the presentations throughout the
text. The exposition on state-space modeling, ARMAX models, and (multivariate)
regression with autocorrelated errors in Chapter 6 have been expanded. In this edition,
we use standard R functions as much as possible, but we use our own scripts (included
in astsa) when we feel it is necessary to avoid problems with a particular R function;
these problems are discussed in detail on the website for the text under R Issues.
We thank John Kimmel, Executive Editor, Springer Statistics, for his guidance
in the preparation and production of this edition of the text. We are grateful to Don
Percival, University of Washington, for numerous suggestions that led to substantial
improvement to the presentation in the second edition, and consequently in this
edition. We thank Doug Wiens, University of Alberta, for help with some of the R
code in Chapter 4 and Chapter 7, and for his many suggestions for improvement of
the exposition. We are grateful for the continued help and advice of Pierre Duchesne,
University of Montreal, and Alexander Aue, University of California, Davis. We also
thank the many students and other readers who took the time to mention typographical
errors and other corrections to the first and second editions. Finally, work on the this
edition was supported by the National Science Foundation while one of us (D.S.S.)
was working at the Foundation under the Intergovernmental Personnel Act.
Davis, CA
Pittsburgh, PA
September 2010
Robert H. Shumway
David S. Stoffer
“tsa4_trimmed” — 2017/12/8 — 15:01 — page ix — #7
Contents
Preface to the Fourth Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
v
Preface to the Third Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vii
1
Characteristics of Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1 The Nature of Time Series Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Time Series Statistical Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Measures of Dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4 Stationary Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5 Estimation of Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.6 Vector-Valued and Multidimensional Series . . . . . . . . . . . . . . . . . . . . .
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
2
8
14
19
26
33
38
2
Time Series Regression and Exploratory Data Analysis . . . . . . . . . . . . .
2.1 Classical Regression in the Time Series Context . . . . . . . . . . . . . . . . .
2.2 Exploratory Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Smoothing in the Time Series Context . . . . . . . . . . . . . . . . . . . . . . . . .
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
47
47
56
67
72
3
ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1 Autoregressive Moving Average Models . . . . . . . . . . . . . . . . . . . . . . .
3.2 Difference Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Autocorrelation and Partial Autocorrelation . . . . . . . . . . . . . . . . . . . . .
3.4 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6 Integrated Models for Nonstationary Data . . . . . . . . . . . . . . . . . . . . . .
3.7 Building ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8 Regression with Autocorrelated Errors . . . . . . . . . . . . . . . . . . . . . . . .
3.9 Multiplicative Seasonal ARIMA Models . . . . . . . . . . . . . . . . . . . . . . .
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
77
77
90
96
102
115
133
137
145
148
156
“tsa4_trimmed” — 2017/12/8 — 15:01 — page x — #8
x
Contents
4
Spectral Analysis and Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1 Cyclical Behavior and Periodicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 The Spectral Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 Periodogram and Discrete Fourier Transform . . . . . . . . . . . . . . . . . . .
4.4 Nonparametric Spectral Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Parametric Spectral Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6 Multiple Series and Cross-Spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7 Linear Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.8 Lagged Regression Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.9 Signal Extraction and Optimum Filtering . . . . . . . . . . . . . . . . . . . . . . .
4.10 Spectral Analysis of Multidimensional Series . . . . . . . . . . . . . . . . . . .
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
167
168
174
181
191
205
208
213
218
223
227
230
5
Additional Time Domain Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.1 Long Memory ARMA and Fractional Differencing . . . . . . . . . . . . . .
5.2 Unit Root Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 GARCH Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4 Threshold Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5 Lagged Regression and Transfer Function Modeling . . . . . . . . . . . . .
5.6 Multivariate ARMAX Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
241
241
250
253
261
265
271
284
6
State Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1 Linear Gaussian Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Filtering, Smoothing, and Forecasting . . . . . . . . . . . . . . . . . . . . . . . . .
6.3 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.4 Missing Data Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.5 Structural Models: Signal Extraction and Forecasting . . . . . . . . . . . .
6.6 State-Space Models with Correlated Errors . . . . . . . . . . . . . . . . . . . . .
6.6.1 ARMAX Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.6.2 Multivariate Regression with Autocorrelated Errors . . . . . . .
6.7 Bootstrapping State Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.8 Smoothing Splines and the Kalman Smoother . . . . . . . . . . . . . . . . . . .
6.9 Hidden Markov Models and Switching Autoregression . . . . . . . . . . .
6.10 Dynamic Linear Models with Switching . . . . . . . . . . . . . . . . . . . . . . .
6.11 Stochastic Volatility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.12 Bayesian Analysis of State Space Models . . . . . . . . . . . . . . . . . . . . . .
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
287
288
292
302
310
315
319
320
322
325
331
334
345
357
365
375
7
Statistical Methods in the Frequency Domain . . . . . . . . . . . . . . . . . . . . .
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 Spectral Matrices and Likelihood Functions . . . . . . . . . . . . . . . . . . . .
7.3 Regression for Jointly Stationary Series . . . . . . . . . . . . . . . . . . . . . . .
7.4 Regression with Deterministic Inputs . . . . . . . . . . . . . . . . . . . . . . . . .
7.5 Random Coefficient Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
383
383
386
388
397
405
“tsa4_trimmed” — 2017/12/8 — 15:01 — page xi — #9
Contents
xi
7.6 Analysis of Designed Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.7 Discriminant and Cluster Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.8 Principal Components and Factor Analysis . . . . . . . . . . . . . . . . . . . . .
7.9 The Spectral Envelope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
407
421
437
453
464
Appendix A Large Sample Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A.1 Convergence Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A.2 Central Limit Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A.3 The Mean and Autocorrelation Functions . . . . . . . . . . . . . . . . . . ...