Confidence Regions for Parameters in Stationary Time Series Models With Gaussian Noise

Abstract

This article develops two new empirical likelihood methods for long-memory time series models based on adjusted empirical likelihood and mean empirical likelihood. By application of Whittle likelihood, one obtains a score function that can be viewed as the estimating equation of the parameters of the long-memory time series model. An empirical likelihood ratio is obtained which is shown to be asymptotically chi-square distributed. It can be used to construct confidence regions. By adding pseudo samples, we simultaneously eliminate the non-definition of the original empirical likelihood and enhance the coverage probability. Finite sample properties of the empirical likelihood confidence regions are explored through Monte Carlo simulation, and some real data applications are carried out.

1 Introduction

The empirical likelihood (EL) method is originally designed to construct a confidence region only for independent data [1,2]. Nowadays, it is quite popular in the statistical inference of time series, (dependent data) thanks to the asymptotical independent property of the periodogram ordinates, see [3], [4], and [5]. EL received considerable attention because of its nice statistic properties. For example, it runs a low risk of a misspecified probability model by setting up a semi-parametric moment model; it is easy to construct a confidence region or interval under the chi-square approximation; there is no need to calculate the covariance estimates when it is used to construct confidence regions; and the shape of confidence region that is naturally driven by data.

In spite of nice properties of EL, it still does not work well for the case of small sample or high-dimensional data. Its main drawback is the large coverage error of the corresponding confidence region. One reason for the under-coverage is that the original EL ratio poorly approximates to the chi-square limiting distribution. This under-coverage issue can be alleviated to some extent by the Bartlett correction [6]. Another reason for the under-coverage is the non-definition problem of the EL ratio. When the sample size is small, the definition of the original EL ratio often does not exist. [7] proposed adjusted EL by adding pseudo points to ensure the original convex hull lie on the opposite of the origin, which not only eliminates the non-definition but also improves the accuracy of approximation with the conventional level of adjustment. [8] put forward the optimal adjustment level to improve the chi-square approximation with a high-order precision, but the problem is not solved. Afterward, [9] derived an adjustment factor from the Bartlett correction and proved by adding two pseudo observations; the new adjusted EL (AEL) has the same order of chi-square approximation as the Bartlett correction. Many research studies about choosing an optimal adjustment factor arose, such as [10], [11], [12], and [13]. In practice, when the dimension of the unknown parameter is large, it is difficult to calculate the Bartlett correction factor. Recently, the mean EL (MEL) was proposed to improve the precision of the EL-based confidence regions by constructing a new pseudo data point [14]. By greatly increasing the sample size, the MEL method leads to a more accurate chi-square approximation. Hence, the corresponding coverage error is reduced, and the coverage probability is enhanced.

For time series, EL inherits the undesirable problem of under-coverage. To the best of our knowledge, there have been some works applying AEL on the improvement of the precision of the EL-based confidence region, such as [15] and [16]. But under their proposed conventional adjustment level a = max (1, log(n)/2), the precision of chi-square approximation distribution of AEL is not obviously enhanced. In this article, we propose another adjustment level to construct new AEL for the parameters in stationary short- and long-memory time series models with Gaussian noise. Following the Liu & Chen’s method, our proposed AEL possesses a high-order approximation to chi-square distribution by adding two pseudo observations. We also propose the MEL method for estimating the parameters of stationary time series models. By increasing the sample size, the MEL does enhance the confidence precision. The Monte Carlo simulation results indicate that our proposed AEL-based confidence regions benefit more accurate coverage probabilities than those of the original EL and the previous AEL. When the sample size is moderate, the coverage probability based on MEL is comparable to the previous AEL. There is no need to compute the Bartlett correction factor for MEL, which is a challenge when the dimension of the parameter vector is large.

The remainder of this article is organized as follows; in Section 2, we present the proposed AEL and MEL ratio statistics for parameters in stationary ARMA and ARFIMA processes and deduce their asymptotical chi-square properties. Monte Carlo simulation results are provided in Section 3 to show the improved finite-sample performance of our methods. Real data examples are presented in Section 4. Section 5 is the brief proof of the theorem.

2 Methodology

In this section, we derive the asymptotical chi-square properties of the new AEL and MEL ratio statistics for the parameters in representative stationary short- and long-memory models, that is, ARMA and ARFIMA models. We begin with introducing the moment-estimating equations of parameters in these models.

2.1 Estimating Equations

Suppose, we have a time series satisfying the relationshipwhere and with ϕ₀ = θ₀ = − 1 and B is the backward operator. The two polynomials have no common factor to avoid the parameter redundancy. All roots of their corresponding equations strictly lie out of the unit circle to make sure that the model is stationary and invertible, which is the most important setup for many time series studies. d is the constant memory parameter and ϵ_t is a Gaussian white noise with mean 0 and variance σ². Then, when d = 0, it is the popular ARMA (p,q) (short-memory) model. When d ∈ (0, 0.5), it is the widely used ARFIMA (p,d,q) (long-memory) model [17]. σ² is often considered as a nuisance parameter. Then, β = (ϕ₁, … , ϕ_p, d, θ₁, … , θ_q) is the parameter vector of our interest with the dimension m = p + q + 1. Many important literatures come up with the application of the fractional long-memory model ([18], [19], and [20]).

By taking the derivative of Whittle likelihood [21], the estimating equations are derived aswhere N = [(T − 1)/2] and [x] is the integer part of x. The periodogram ordinates are denoted asand the spectral density function is

2.2 AEL of β

It is well known that when the origin lies out of certain convex constraints involving the computation of EL, the solution of the optimization problem does not exist, which results in the no definition of EL and the under-coverage of the corresponding confidence region. In this section, we propose a new AEL in time series to ensure the well definedness of AEL and improvement of the coverage probability of confidence regions.

For simplicity, denote ψ_i≔ψ_i (I (ω_i), β). Based on the original sample set , for a given β, the empirical log-likelihood ratio is defined as

If the origin contains in the convex set Ω_β, by a simple Lagrange multiplier calculation, and the Lagrange multiplier λ_β is the solution to . Therefore, the level of the 1 − α confidence region is constructed aswhere is the 1 − α quantile of chi-square distribution with the degree m of freedom. Such EL-based confidence regions often suffer from the problem of under-coverage.

To overcome such drawback, we propose a new AEL following the Liu & Chen’s method. By adding two pseudo observations, the AEL not only guarantees the likelihood ratio to be always well defined but also obviously improves the coverage probability of the confidence region in stationary time series. Set the two pseudo observations and , where a₁, a₂ are positive and . Then, for each given β, the new AEL ratio statistic is defined as

By the Lagrange method, we havewhere the Lagrange multiplier satisfies the equation

[22] proved such AEL ratio statistic R^A (β₀) approximated to chi-square distribution with order O (n⁻²) for the independent sample when β₀ is the true value. Here, we assert that such result is preserved for the stationary time series model as the following theorem:

Theorem 2.1. Assume the characteristic function of ψ satisfies ’s condition,AlsoE‖ψ‖¹⁸ < ∞andvar(ψ) are positive definites. Ifβ₀is the true value, then

Consequently, the 1 − α confidence region for β based on AEL is constructed as whose coverage error is O (n⁻²).

Remark 1. The proof of Theorem 2.1 is similar to that of Theorem 1 in [22], hence is omitted. The two positive adjustment factors a₁ and a₂ are also obtained following the way of [22]. That is, a₁ and a₂ originate from the Bartlett correction factor. It is the intrinsic relationship between the new AEL and the Bartlett-corrected EL that makes the precision of approximation to enhance obviously. In practice, a₁ and a₂ are replaced by their moment estimators, which do not affect the order of chi-square approximation. For more details, refer to [22].

2.3 MEL ofβ

When the dimension of the parameter vector m ≥ 3, it is difficult to compute the Bartlett correction factor. To avoid the computation and to resolve the under-coverage problem, we derive the MEL method in this subsection. By greatly increasing the sample size, MEL is constructed on the pseudo sample set . For simplicity, we denote with g_k≔(ψ_i + ψ_j)/2 and K = N(N + 1)/2.

Then, the MEL ratio for given β is defined as

By a simple Lagrange calculation, we haveand the Lagrange multiplier is the solution to

Theorem 2.2. Under the assumptions of A1–A4 [4], ifβ₀is the true value,in distribution asn → ∞.

Consequently, the MEL-based confidence region for β of level 1 − α is

In the next section, we will verify the accurate coverage probability of the confidence region under the finite sample by simulation study for different versions of EL.

3 Simulation

To investigate the finite-sample performance of our proposed AEL_*(the notation of our proposed AEL in the following statements) and MEL, we carry out extensive Monte Carlo simulation studies of the ARMA (p,q) and ARFIMA (p,d,q) models in this section. To emphasize that our proposed adjustment level is better than the conventional adjustment levela = max{ log(N)/2, 1} of AEL of [15] and [16], we also studied the unadjusted EL and their AEL. Furthermore, we studied the confidence region graphically.

3.1. Simulation Setup

We consider several ARMA (p,q) and ARFIMA (p,d,q) processes with different sample sizes and Gaussian noise with zero mean. In each case, 5,000 replications are generated to compute the coverage probability. Nominal levels are set to be 1 − α = 0.90, 0.95, 0.99, respectively. It is notable that although the series length is T, the sample size we use is only N = [(T − 1)/2]. In the simulations, we use the consistent estimators and to replace the adjustment factors a₁ and a₂, respectively. The computations and are completely similar to those of Section 3.3 in [22]. When the dimension of the parameter m = 1, we only add one pseudo observation with the adjustment level . Specifically,whereα^rs⋯t = E (Y^rY^s … , Y^t),Y^tis thetth component ofY, Y = P^τψ₀, ψ₀ = ψ(I(ω), β₀), and P is the orthogonal matrix such that.{ξ_i, i = 1, … , m}are eigenvalues of. Note. Then, the consistent estimators and are obtained by replacing the components ofa₁, a₂with their corresponding consistent estimators, which are given in the following table:

3.2 Simulation Results

Tables 1–7 report the coverage probabilities of confidence regions based on four versions of EL. First, we find that our proposed AEL_* performs better than other ELs in terms of the coverage probability for all cases and that the coverage probability of our proposed AEL_* is the closest to the normal level. Second, when the sample size is very small, the coverage probability based on MEL is smaller than that based on AEL with the adjustment level a = max (log(N)/2, 1). But, when the sample size is moderate, the coverage probability is comparable to that based on AEL method, which is because the improvement only relies on the increasing sample size in essence. So, MEL enhances the coverage probability at some expense of computational efficiency. Third, the coverage probabilities based on AEL and AEL_* are not always enhanced with the increasing sample size. Fourth, from Table 1, when the coverage probability increases, the corresponding average length of the confidence interval is getting large. Fifth, Table 3 indicates that the coverage probabilities become small when the parameter approximates the critical value tending to non-stationarity. The confidence regions in Figures 1, 2 are respectively depicted as the case with the series length 1,000 and 200. When the sample size is large, there is little difference in four kinds of the confidence region in Figure 1. In Figure 2, obviously, when the sample size is small, our proposed AEL_*-based confidence region is the smallest among the four counterparts, and the MEL-based confidence contour contains others. It indicates that our proposed AEL_* has not only high-coverage probabilities but also small confidence regions, and MEL has high-coverage probabilities and relative large confidence regions. The shape of the confidence region matches with the data-driven property of the EL method. That is, the shapes of confidence regions are completely determined by the data.

FIGURE 1

FIGURE 2

4 Real Examples

In this section, we illustrate and compare the validity of our proposed EL methods described in previous sections by analyzing some real examples.

4.1 Annual U.S. Unemployment Rate

First, we take annual U.S. unemployment rate series as an example to investigate the confidence region of our proposed EL methods. The data are collected from 1948 to 2019 and available from https://forecast-chart.com/forecast-unemployment-rate.html. The trajectory of these data is displayed in Figure 3A. We consider it as a realization of a stationary process. By the sample autocovariance function (ACF) and partial autocovariance function (PACF) analysis, we fit the unemployment rate by an AR (2) model. The 95% confidence regions based on our proposed AEL_* and MEL are displayed in Figure 3B. The shapes coincide with EL’s data-driven property, and the size indicates our proposed methods are much better than the previous EL and AEL methods.

FIGURE 3

4.2 S&P 500 VIX

S&P 500 VIX is a forward-looking index. If it is extended to the price observations of the broader market level index, the investor will get a peek into volatility of the larger market. So, it is meaningful to fit a proper model. Here, we collect the data from April 25, 2019 to November 21, 2019. The trajectory is displayed in Figure 4A. We fit the data by an ARFIMA (1,d, 0) model with maximum likelihood point estimates . Then, the confidence regions are exhibited in Figure 4B. Compared with the original EL and the previous AEL, our proposed AEL_*-based confidence region is still the best, and the MEL-based confidence contour also contains the others.

FIGURE 4

4.3 Shanghai Securities Composite Index

Finally, we analyze the daily log-return rate of Shanghai Securities Composite Index. The data range from June 4, 2020 to March 26, 2021. Figure 5A displays the realization of the index. We fit it as an ARFIMA (0,d, 0) model. The maximum likelihood and the EL estimate are and , respectively, and the lengths of four kinds of the EL confidence interval are displayed in Table 8. We find that the confidence interval based on MEL is still the largest, and the one based on our proposed AEL_* is still the smallest.

FIGURE 5

5 Conclusion

In this article, we introduce two new versions of EL to construct confidence regions for parameters in stationary short- and long-memory time series. Our proposed AEL_* and MEL do enhance the approximation precision of chi-square limiting distribution, which determines the good performance of corresponding confidence regions. Simulations show that our proposed AEL_* has the better coverage probability than that of the previous AEL and MEL.

References

• 1.

  OwenABEmpirical Likelihood Ratio Confidence Intervals for a Single Functional. Biometrika (1988) 75:237–49. 10.1093/biomet/75.2.237

  • CrossRef
  • Google Scholar

• 2.

  OwenABEmpirical Likelihood Ratio Confidence Regions. Ann Stat (1990) 18:90–120. 10.1214/aos/1176347494

  • CrossRef
  • Google Scholar

• 3.

  MontiAEmpirical Likelihood Confidence Regions in Time Series Models. Biometrika (1997) 84:395–405. 10.1093/biomet/84.2.395

  • CrossRef
  • Google Scholar

• 4.

  NordmanDJLahiriSNA Frequency Domain Empirical Likelihood for Short- and Long-Range Dependence. Ann Stat (2006) 34:3019–50. 10.1214/009053606000000902

  • CrossRef
  • Google Scholar

• 5.

  YauCYEmpirical Likelihood in Long-Memory Time Series Models. J Time Ser Anal (2012) 33:269–75. 10.1111/j.1467-9892.2011.00756.x

  • CrossRef
  • Google Scholar

• 6.

  DiCiccioTJHallPRomanoJPEmpirical Likelihood Is Bartlett Correctable. Ann Stat (1991) 19:1053–61. 10.1214/aos/1176348137

  • CrossRef
  • Google Scholar

• 7.

  ChenJVariyathAMAbrahamBAdjusted Empirical Likelihood and its Properties. J Comput Graphical Stat (2008) 17:426–43. 10.1198/106186008x321068

  • CrossRef
  • Google Scholar

• 8.

  EmersonSCOwenABCalibration of the Empirical Likelihood Method for a Vector Mean. Electron J Stat (2009) 3:1161–92. 10.1214/09-ejs518

  • CrossRef
  • Google Scholar

• 9.

  ChanNHLiuLBartlett Correctability of Empirical Likelihood for Time Series. Jjss (2010) 40:221–38. 10.14490/jjss.40.221

  • CrossRef
  • Google Scholar

• 10.

  LiXChenJHWuYHTuDSConstructing Nonparametric Likelihood Confidence Regions with High Order Precisions. Stat Sinica (2011) 21:1767–83. 10.5705/ss.2009.117

  • CrossRef
  • Google Scholar

• 11.

  ChengCLiuYLiuZZhouWBalanced Augmented Jackknife Empirical Likelihood for Two Sample U-Statistics. Sci China Math (2018) 61(6):1129–38. 10.1007/s11425-016-9071-y

  • CrossRef
  • Google Scholar

• 12.

  DahlhausRWefelmeyerWAsymptotically Optimal Estimation in Misspecified Time Series Models. Ann Stat (1996) 24:952–74. 10.1214/aos/1032526951

  • CrossRef
  • Google Scholar

• 13.

  JiangFWangLAdjusted Blockwise Empirical Likelihood for Long Memory Time Series Models. Stat Methods Appl (2018) 27:319–32. 10.1007/s10260-017-0403-1

  • CrossRef
  • Google Scholar

• 14.

  LiangWDaiHHeSMean Empirical Likelihood. Comput Stat Data Anal (2019) 138:155–69. 10.1016/j.csda.2019.04.007

  • CrossRef
  • Google Scholar

• 15.

  Piyadi GamageRDNingWGuptaAKAdjusted Empirical Likelihood for Long-Memory Time-Series Models. J Stat Theor Pract (2017) 11:220–33. 10.1080/15598608.2016.1271373

  • CrossRef
  • Google Scholar

• 16.

  Piyadi GamageRDNingWGuptaAKAdjusted Empirical Likelihood for Time Series Models. Sankhya B (2017) 79:336–60. 10.1007/s13571-017-0137-y

  • CrossRef
  • Google Scholar

• 17.

  GrangerCWJJoyeuxRAn Introduction to Long-Memory Time Series Models and Fractional Differencing. J Time Ser Anal (1980) 1:15–29. 10.1111/j.1467-9892.1980.tb00297.x

  • CrossRef
  • Google Scholar

• 18.

  LiMMulti-fractional Generalized Cauchy Process and its Application to Teletraffic. Physica A: Stat Mech its Appl (2020) 550:123982. 10.1016/j.physa.2019.123982

  • CrossRef
  • Google Scholar

• 19.

  LiMGeneralized Fractional Gaussian Noise and its Application to Traffic Modeling. Physica A (2021) 579:1236137. 10.1016/j.physa.2021.126138

  • CrossRef
  • Google Scholar

• 20.

  LiMModified Multifractional Gaussian Noise and its Application. Phys Scr (2021) 96:125002. 10.1088/1402-4896/ac1cf6

  • CrossRef
  • Google Scholar

• 21.

  WhittlePEstimation and Information in Stationary Time Series. Ark. Mat.r Matematik (1953) 2:423–34. 10.1007/bf02590998

  • CrossRef
  • Google Scholar

• 22.

  LiuYKChenJHAdjusted Empirical Likelihood with High-Order Precision. Ann Stat (2010) 38(3):1341–62. 10.1214/09-aos750

  • CrossRef
  • Google Scholar