Using estimating equations to estimate unknown parameters was widely studied both in empirical applications and theoretical studies, see Hall [3] for reviews of this method. If the number of estimating equations is identical to the number of unknown parameters (just-identification), we can use usual moments method to estimate the unknown parameters and further study its statistical asymptotic behavior. But in applications, especially in econometrics time series datas and longitudinal datas analysis, it is often the case that the number of estimating equations is larger than the number of unknown parameters (called over-identification) and in this case the solution of moments estimating equations does not exists in general. To deal with this problem, several estimation methods were proposed in the literature, the most popular were the generalized method of moments (GMM) by Hansen in [4] and generalized empirical likelihood (GEL) by Smith in [8]. This paper mainly focused on the GMM estimators.

To fix the main idea of GMM, suppose we have observed a random sample $(X_1,\cdots,X_n)$ from some population $X$ with unknown parameters $\theta\in\Theta$. The true value $\theta_0$ satisfies the population moment conditions

${\bf{E}}[g(X,{\theta _0})] = 0,$

(1.1)

where $\Theta$ is a compact such set of $\mathbb{R}^p,p\in\mathbb{N}_+$ and $g$ is a $\mathbb{R}^q,q\in\mathbb{N}_+$ valued measurable function. In the over identified case, i.e., $q>p$, Hansen (see [4]) introduced the GMM estimator of $\theta_0$ as

$\begin{equation} \hat{\theta}_{1n}=\arg \min\limits_{\theta\in\Theta}\left( \frac{1}{n}\sum\limits_{i=1}^ng(X_i,\theta)\right)'W_n \left(\frac{1}{n}\sum\limits_{i=1}^ng(X_i,\theta)\right),\end{equation}$

(1.2)

where $'$ means transposition operation of matrix and $W_n$ is a positive semi-definite weighting matrix that converges in probability to a positive definite constants matrix $W$. If $W_n\equiv I$, then the $\hat{\theta}_{1n}$ is the point of $\theta$ so that minimize the sample covariance matrix. It is known that if the model is not misspecified and under some mild regularity conditions, Hansen (see [4]) established the strong consistency and asymptotic normality of the GMM estimator. Precisely, he proved

$\begin{equation} \sqrt{n}(\hat{\theta}_{1n}-\theta_n)\to_d N(0,V_W),\quad V_W=(G'WG)^{-1}G'W\Sigma WG(G'WG)^{-1},\end{equation}$

(1.3)

where $G = {\bf{E}}(\nabla {g_\theta }(X,{\theta _0})),\Sigma = {\bf{E}}[g(X,{\theta _0})g'(X,{\theta _0})]$. Observing that the asymptotic variance $V_W$ depends on $W$. When constructing the confidence intervals of $\theta_0$, we hope $V_W$ as small as possible (in the positive semi-definite sense). It is easy to find that $V_W$ is minimized at $V_W=\Sigma^{-1}$. Since $\Sigma$ is unknown, we can use $\hat{\theta}_{1n}$ as a preliminary estimator, and naturally defined

${\Sigma}_n^{-1}=\left(\frac{1}{n}\sum\limits_{i=1}^ng(X_i,\hat{\theta}_{1n})g(X_i,\hat{\theta}_{1n})' \right)^{-1} $

as a estimator of $\Sigma^{-1}$. Then we drive a two-step GMM estimator:

$\begin{equation} \hat{\theta}_{2n}=\arg \min\limits_{\theta\in\Theta}\left(\frac{1}{n}\sum\limits_{i=1}^ng(X_i,\theta)\right)' \hat{\Sigma}^{-1}_n \left(\frac{1}{n}\sum\limits_{i=1}^ng(X_i,\theta)\right),\end{equation}$

(1.4)

and for convenience we call $\hat{\theta}_{1n}$ as one-step GMM estimator.

The large sample behaviors of one-step and two-step has been studied extensively. Under mild conditions, $\hat{\Sigma}_n^{-1}$ converges weakly to ${\Sigma}^{-1}$ and $\hat{\theta}_{2n}$ is consistent and asymptotic normality (see [2]). Otsu studied the large deviation principle of the these estimators in [5] and the moderate deviation principle was also obtained in [6]. This paper is devoted to study the law of iterated logarithm for GMM estimators.

The paper is organized as follows: In Section 2, we state the basic assumptions and main results of the paper, the proof of the main results are given in Section 3.

We first give some notations. For any $m\times n$ matrix $A=(a_{ij})_{m\times n}$, we write $A'$ as its transposition, and define its norm as

$\|A\|=\sqrt{\mathrm{tr}(AA')}=\left(\sum\limits_{i=1}^m\sum\limits_{j=1}^n a_{ij}^2\right)^{1/2}.$

Denote

$\begin{array}{l} G(X,\theta ) = \nabla {g_\theta }(X,\theta ),\qquad G = {\bf{E}}\left( {\nabla {g_\theta }(X,{\theta _0})} \right),\\ {{\hat g}_n}(\theta ) = \frac{1}{n}\sum\limits_{i = 1}^n g ({X_i},\theta ),\qquad {{\hat G}_n}(\theta ) = \frac{1}{n}\sum\limits_{i = 1}^n G ({X_i},\theta ),\\ Q({\theta _0}) = {({q_{ij}}({\theta _0}))_{p \times p}} = G'WG \end{array}$

and $h_{i}(\theta_0)=(h_{ij}(\theta_0))_{p\times 1}:=G'Wg(X_i,\theta_0)$, in the following we assume $W$ is positive definite constants matrix, so $Q(\theta_0)$ is positive, we can define $A(\theta_0)=(a_{ij}(\theta_0))_{p\times p}= -Q(\theta_0)^{-1}$, which is a negative definite matrix.

Following are the assumptions of the paper.

Assumption 2.1 (1) $\Theta$ is compact and $\theta_0\in \mathrm{int}(\Theta)$. There exists a measurable function $L: \mathbb{R}\rightarrow [0,\infty)$ so that for a.e. $x$, $\|g(x,\theta_1)-g(x,\theta_2)\|\leq L(x)\|\theta_1-\theta_2\|,\forall \theta_1,\theta_2\in\Theta$. For some $\delta_1>0$ and all $\theta\in\Theta$, ${\bf{E}}\|g(X,\theta)\|^{2+\delta_1}<\infty$ and ${\bf{E}}L(X) < \infty $.

(2) There exists a measurable function $H: \mathbb{R} \rightarrow[0,\infty)$ and $\delta_2>0$ such that $\|G(x,\theta)-G(x,\theta_0)\|\leq H(x)\|\theta-\theta_0\|$ holds for a.e. $x$ and $\theta\in U(\theta_0,\delta_2):=\{\theta:\|\theta-\theta_0\|<\delta_2\}$. Besides, we assume ${\bf{E}} H(X)<\infty$, ${\bf{E}} H(X)^2<\infty$, ${\bf{E}}\|G(X,\theta)\|<\infty$ and ${\bf{E}}\|G(X,\theta)\|^2<\infty$, for all $\theta\in\Theta$. The $(q\times p) $matrix $G$ has the full column rank and $\Sigma$ is positive definite.

For the weighting matrix $W_n$, we assume

Assumption 2.2 $\{W_n\}_{n\geq 1}$ is a sequence of positive semi-definite matrices, and $W_n$ converges weakly to $W$ in the sense of matrix norm, where $W$ is a positive definite matrix of constants. Besides, $\sup_n{\bf{E}}\|W_n\|<\infty$.

Suppose that Assumptions 2.1, 2.2 hold, our main results are

Theorem 2.1 Let $\hat{\theta}_{1n}=(\hat{\theta}_{11},\hat{\theta}_{21},\cdots,\hat{\theta}_{p1})'$ and $\hat{\theta}_{2n}=(\hat{\theta}_{12},\hat{\theta}_{22},\cdots,\hat{\theta}_{p2})'$ be the one-step and two-step GMM estimators, then for any $\theta_0=(\theta_{10},\theta_{20}\cdots,\theta_{p0})'\in\Theta$, we have

$\begin{equation} \limsup\limits_{n\rightarrow\infty}\frac{n}{\sqrt{2n\log\log n}} |\hat{\theta}_{si}-\theta_{s0}|=\varphi(s),s=1,2,\cdots,p,i=1,2,\end{equation}$

(2.1)

where

$\begin{equation*} \varphi^2(s)=\sum\limits_{m=1}^p \sum\limits_{k=1}^p a_{sm}(\theta_0)a_{sk}(\theta_0){\bf{E}}\left(h_{im}(\theta_0)h_{ik}(\theta_0)\right),s=1,2,\cdots,p. \end{equation*}$

Remark 2.1 Hansen (see [4]) established the strong consistency of the GMM estimator. Under Assumptions 2.1, 2.2 and classical strong law of large numbers we have

$\begin{align} & P\left( \underset{\delta \to 0}{\mathop{\lim }}\,\underset{\begin{matrix} n\to \infty \\ \theta \in U({{\theta }_{0}},\delta ) \\ \end{matrix}}{\mathop{\lim \sup }}\,\|{{{\hat{G}}}_{n}}(\theta )-{{{\hat{G}}}_{n}}({{\theta }_{0}})\|=0 \right)=1, \\ & \|{{{\hat{G}}}_{n}}({{\theta }_{0}})-G\|\to 0,\qquad {{{\hat{G}}}_{n}}({{{\hat{\theta }}}_{1n}}{)}'W{{{\hat{g}}}_{n}}({{\theta }_{0}})\to 0\rm{ a}\rm{.s}\rm{.}n\to \infty . \\ \end{align}$

Because the independence is necessary in proof of the next lemma and theorem, we first suppose $W_n=W$. Then under the strong limit condition of $W_n$ and W, we get the results for $W_n$.

Lemma 3.1 (see [7], Theorem 4, Chap. X) Let $\{U_i:i\geq1\}$ be a sequence of independent random variables with ${\bf{E}} U_i=0,i\geq1,{\bf{E}} U_i^2<\infty$. Setting $S_n^2=\sum\limits_{i=1}^n{\bf{E}} U_i^2$, if $\{U_i,i\geq 1\}$ satisfies the conditions

$\begin{equation} \liminf\limits_{n\rightarrow\infty}\frac{S_n^2}{n}>0,\end{equation}$

(3.1)

and for some $\delta>0$,

$\begin{equation} \limsup\limits_{n\rightarrow\infty}\frac{1}{n}\sum\limits_{k=1}^n {\bf{E}}[U_k^2|\log|U_k||^{1+\delta}]<\infty. \end{equation}$

(3.2)

Then

$\begin{equation} \limsup\limits_{n\rightarrow\infty}\frac{|\sum\limits_{i=1}^nU_i|}{\sqrt{2S_n^2\log\log S_n^2}}=1\qquad \mbox{a.s..} \end{equation}$

(3.3)

Remark 3.1 The following easy verified condition is sufficient for (3.2),

$\begin{equation*} \limsup\limits_{n\rightarrow\infty}\frac{1}{n}\sum\limits_{k=1}^n {\bf{E}}(|U_k|^{2+\delta})<\infty. \end{equation*}$

Since for $x$ large enough, we have elementary inequality $(\log x)^{1+\delta}<x^\delta$ for any $\delta>0$.

Lemma 3.2 We denote

$\begin{eqnarray*} \omega_i(s) = \sum\limits_{m=1}^p a_{sm}(\theta_0)h_{im}(\theta_0) \end{eqnarray*}$

and $S_n^2=\sum\limits_{i=1}^n {\bf{E}} \omega_i^2(s)$. Then

$\begin{equation} \limsup\limits_{n\rightarrow\infty}\frac{|\sum\limits_{i=1}^n\omega_i(s)|}{\sqrt{2S_n^2\log\log S_n^2}}=1\qquad \mbox{a.s..} \end{equation}$

Proof Observe that ${\bf{E}}(h_{im}(\theta_0))=G'W {\bf{E}}[g(X_i,\theta_0)]=0$ and $\{h_{im},i\geq1\}$ is a sequence of independent random variables. Consequently,

$\begin{equation*} {\bf{E}}\omega_i(s)=\sum\limits_{m=1}^p a_{sm}(\theta_0){\bf{E}}(h_{im}(\theta_0))=0. \end{equation*}$

By virtue of Assumptions 2.1, 2.2, we have

$\begin{equation*} \sup\limits_{i\geq1}\max\limits_{1\leq m\leq p}{\bf{E}}\|h_{im}\|^{2+\delta} \leq\sup\limits_{i\geq1}{\bf{E}}\|GWg(X_i,\theta_0)\|^{2+\delta} <\infty. \end{equation*}$

Therefore

$\begin{align*} \sup\limits_{i\geq1}{\bf{E}}|\omega_i(s)|^{2+\delta}<\infty,\qquad \limsup\limits_{n\rightarrow\infty}\frac{1}{n}\sum\limits_{i=1}^n {\bf{E}}|\omega_i(s)|^{2+\delta}<\infty. \end{align*}$

On the other hand, it is obviously that $ \lim\limits_{n\rightarrow\infty}\frac{S_n^2}{n}={\bf{E}} \omega_1^2(s)>0$, consequently, it follows from Lemma 3.1 that (3.4) holds.

Now it ready to give the proof of Theorem 2.1.

For the consistence between theorem and lemma, we set first

$\begin{equation} \theta_n(t)=(1-t)\theta_0+t\hat{\theta}_{1n},\qquad u_n(t)=\hat{G}_n(\hat{\theta}_{1n})'W\hat{g}_n(\theta_n(t)),\qquad 0\leq t\leq1. \end{equation}$

(3.5)

Hence $\theta_n(1)=\hat{\theta}_{1n}$, $u_n(1)=\hat{G}_n(\hat{\theta}_{1n})'W\hat{g}_n(\hat{\theta}_{1n})=0.$ From (3.5), we have

$\begin{equation} u_n(0)=u_n(1)-\int_0^1 u_n'(t)dt=-\int_0^1 Q_n(\theta_n(t))(\hat{\theta}_{1n}-\theta_0)dt,\end{equation}$

(3.6)

where

$\begin{equation*} Q_n(\theta_n(t))=\hat{G}_n(\hat{\theta}_{1n})'W\hat{G}_n(\theta_n(t)). \end{equation*}$

We denote

$\begin{equation} \varphi^2(s)=\lim\limits_{n\rightarrow\infty}\frac{S_n^2}{n} =\sum\limits_{m=1}^p \sum\limits_{k=1}^p a_{sm}(\theta_0)a_{sk}(\theta_0){\bf{E}}\left(h_{im}(\theta_0)h_{ik}(\theta_0)\right). \end{equation}$

(3.7)

It follows from Remark 2.1 and (3.5) that $\|\theta_n(t)-\theta_0\|\rightarrow0$ a.s. for all $t\in (0,1)$. By Remark 2.1 and Assumptions 2.1, 2.2, we have

$\begin{align} & \underset{n\to \infty }{\mathop{\lim \sup }}\,\|{{{\hat{G}}}_{n}}({{{\hat{\theta }}}_{1n}}{)}'W{{{\hat{G}}}_{n}}({{\theta }_{n}}(t))-{G}'WG\| \\ & \le \underset{n\to \infty }{\mathop{\lim \sup }}\,\{\|{{({{{\hat{G}}}_{n}}({{{\hat{\theta }}}_{1n}})-{{{\hat{G}}}_{n}}({{\theta }_{0}}))}^{\prime }}W{{{\hat{G}}}_{n}}({{\theta }_{n}}(t))\|+\|{{({{{\hat{G}}}_{n}}({{\theta }_{0}})-G)}^{\prime }}W{{{\hat{G}}}_{n}}({{\theta }_{n}}(t))\| \\ & +\|({G}'W({{{\hat{G}}}_{n}}({{\theta }_{n}}(t))-{{{\hat{G}}}_{n}}({{\theta }_{0}}))\|+\|{G}'W({{{\hat{G}}}_{n}}({{\theta }_{0}})-G)\|\}\to 0, \\ \end{align}$

almost surely. It follows immediately from above that

$\begin{equation} \lim\limits_{ n\rightarrow\infty}\int_0^1 Q_n(\theta_n(t))dt=\int_0^1 \lim\limits_{n\rightarrow\infty}Q_n(\theta_n(t))dt=\int_0^1 Q(\theta_0)dt=Q(\theta_0). \end{equation}$

(3.8)

Denote

$\begin{equation} \Delta_n:=Q^{-1}(\theta_0)-\left(\int_0^1 Q_n(\theta_n(t))dt \right)^{-1}. \end{equation}$

(3.9)

Since $Q(\theta_0)$ is positive definite, thus $\|Q^{-1}(\theta_0)\|<\infty$. And it follows from (3.8) that there exists $N>0$ so that

$\begin{equation*} \sup\limits_{n>N}\left\|\left(\int_0^1 Q_n(\theta_n(t))dt \right)^{-1}\right\|<\infty. \end{equation*}$

Consequently,

$\|\Delta_n\|\leq\|Q^{-1}(\theta_0)\|\|Q(\theta_0)-\int_0^1Q_n(\theta_n(t))dt\|\|(\int_0^1 Q_n(\theta_n(t))dt )^{-1}\|.$

When $\|\hat{\theta}_{1n}-\theta_0\|\rightarrow0,n\rightarrow\infty$, we get $\Delta_n\rightarrow0.$ From Remark 2.1 and Assumptions 2.1, 2.2, it follows that

$\|{{u}_{n}}(0)-{G}'W{{\hat{g}}_{n}}({{\theta }_{0}})\|=\|{{({{\hat{G}}_{n}}({{\hat{\theta }}_{1n}})-{{\hat{G}}_{n}}({{\theta }_{0}}))}^{\prime }}W{{\hat{g}}_{n}}({{\theta }_{0}})+{{({{\hat{G}}_{n}}({{\theta }_{0}})-G)}^{\prime }}W{{\hat{g}}_{n}}({{\theta }_{0}})\|\to 0$

almost surely. Next, from (3.6) and (3.9), we have

$\begin{align} & \frac{1}{\sqrt{2n\log \log n}}({{{\hat{\theta }}}_{1n}}-{{\theta }_{0}})=\frac{1}{\sqrt{2n\log \log n}}{{\left( -\int_{0}^{1}{{{Q}_{n}}}({{\theta }_{n}}(t))dt \right)}^{-1}}{{u}_{n}}(0) \\ & =\frac{1}{\sqrt{2n\log \log n}}({{\Delta }_{n}}-{{Q}^{-1}}({{\theta }_{0}})){{u}_{n}}(0) \\ & =-\frac{1+o(1)}{\sqrt{2n\log \log n}}{{Q}^{-1}}({{\theta }_{0}}){G}'W{{{\hat{g}}}_{n}}({{\theta }_{0}}). \\ \end{align}$

Thus for any $s=1,\cdots,p$, by virtue of Lemma 3.2, we finally have

$\begin{align} & \underset{n\to \infty }{\mathop{\lim \sup }}\,\frac{1}{\sqrt{2n\log \log n}}n|{{{\hat{\theta }}}_{s1}}-{{\theta }_{s0}}| \\ & =\underset{n\to \infty }{\mathop{\lim \sup }}\,\frac{1}{\sqrt{2n\log \log n}}n\left| ({{a}_{s1}}({{\theta }_{0}}),\cdots ,{{a}_{sp}}({{\theta }_{0}}))\left( \frac{1}{n}\sum\limits_{i=1}^{n}{{{G}'}}Wg({{X}_{i}},{{\theta }_{0}}) \right) \right| \\ & =\underset{n\to \infty }{\mathop{\lim \sup }}\,\frac{1}{\sqrt{2n\log \log n}}\left| \sum\limits_{i=1}^{n}{\sum\limits_{m=1}^{p}{{{a}_{sm}}}}({{\theta }_{0}}){{h}_{im}}({{\theta }_{0}}) \right| \\ & =\underset{n\to \infty }{\mathop{\lim \sup }}\,\sqrt{\frac{S_{n}^{2}\log \log S_{n}^{2}}{n\log \log n}}\sqrt{\frac{1}{2S_{n}^{2}\log \log S_{n}^{2}}}\left| \sum\limits_{i=1}^{n}{{{\omega }_{i}}}(s) \right|=\varphi (s). \\ \end{align}$

This complete the whole proof.

Now we consider the second step GMM estimator $\hat{\theta}_{2n}$ in (1.4). We suppose the new assumption

Assumption 3.1 There exists $\delta_3>0,$ such that ${\bf{E}}\|g(X,\theta_0)g(X,\theta_0)'\|^{2+\delta_3}<\infty $, and ${\bf{E}}\|L(X)^2\|^{2+\delta_3}<\infty$.

It follows from Assumption 3.1 and strong law of large numbers that

$\begin{equation*} \hat{\Sigma}_n\stackrel{{\rm a.s.}}{\longrightarrow}{\bf{E}}(g(X,\theta_0)g(X,\theta_0)'). \end{equation*}$

(3)

So we have $W_n=(\hat{\Sigma}_n)^{-1}\stackrel{{\rm a.s.}}{\longrightarrow}\Sigma^{-1}=W$. We replace $\hat{\theta}_{1n},W$ with $ \hat{\theta}_{2n},W_n$. For $\hat{\theta}_{2n}=(\hat{\theta}_{12},\hat{\theta}_{22},\cdots,\hat{\theta}_{p2})'$, (2.1) holds.