Hypergeometric functions play important roles in harmonic analysis over pseudo-Rieman nian symmetric spaces. Hyperbolic spaces are examples of pseudo-Riemannian symmetric spaces. There are a lot of work on hyperbolic spaces such as [3, 4]. Using a geometric method, Faraut obtained a second order differential equation in the explicit case of hyperbolic spaces ${\rm U}(p, q; {\mathbb F})/{\rm U}(1; {\mathbb F})\times{\rm U}(p-1, q; {\mathbb F})$ with ${\mathbb F}={\mathbb R}, {\mathbb C}$ or $\mathbb H$ in [3]. Later in an algebraic way, i.e., through Casimir operator of ${\mathfrak {sl}}(n+1, {\mathbb R})$, van Dijk and Kosters obtained a hypergeometric equation on the pseudo-Riemannian smmetric space ${\rm SL}(n+1, {\mathbb R})/{\rm GL}(n, {\mathbb R})$ in [5].

A natural extension of [3, 5] is harmonic analysis on the sections of vector bundles over pseudo-Riemannian symmetric spaces. Charchov obtained a hypergeometric equation on the sections of line bundles over complex hyperbolic spaces ${\rm U}(p, q; {\mathbb C})/{\rm U}(1; {\mathbb C})\times{\rm U}(p-1, q; {\mathbb C})$ in his doctor thesis [1]. The differential equation in [1] is the same as the one in [3]. In this paper we will follow the method in [6] to obtain the hypergeometric equation on the sections of line bundles over ${\rm SL}(n+1, {\mathbb R})/{\rm GL}(n, {\mathbb R})$. When the parameter $\lambda$ is zero, our result degenerates to the differential equation in [5]. Our hypergeometric equation will be used to obtaining the Plancherel formula on the sections of the line bundle over ${\rm SL}(n+1, {\mathbb R})/S({\rm GL}(1, {\mathbb R})\times{\rm GL}(n, {\mathbb R}))$ in a future paper.

Let $G={\rm SL}(n+1, {\mathbb R})$ and $H_1={\rm SL}(n, {\mathbb R})$. We imbed $H_1$ in $G$ as usual, i.e., for any $h\in H_1$, $h\mapsto\left(\begin{array}{cc}1& \\ &h \end{array}\right)\in G$. Let $H$ be the subgroup of $G$:

$H=S({\rm GL}(1, {\mathbb R})\times {\rm GL}(n, {\mathbb R}))=\left\{ \left(\begin{array}{cc}\det h^{-1}& \\ &h \end{array}\right): h\in {\rm GL}(n, {\mathbb R})\right\}.$

In what follows $^tA$ denotes the transpose of a matrix $A$. Let $X_1$ be the algebraic manifold of

${\mathbb R}_*^{n+1}\times{\mathbb R}_*^{n+1} ({\mathbb R}_*^{n+1}={\mathbb R}_*^{n+1}\setminus\{\bf0\})$

defined by

$X_1=\{(x, y)\in{\mathbb R}_*^{n+1}\times{\mathbb R}_*^{n+1}: \langle x, y\rangle=x_0y_0+x_1y_1+\cdots+x_ny_n=1\}, $

where $x={^t(}x_0, x_1, \cdots, x_n)$, $y={^t(}y_0, y_1, \cdots, y_n)$, $G$ acts on ${\mathbb R}_*^{n+1}\times{\mathbb R}_*^{n+1}$ by

$\begin{equation}g\cdot(x, y)=(gx, {^tg}^{-1}y)\end{equation}$

(2.1)

for any $g\in G$ and any $(x, y)\in{\mathbb R}_*^{n+1}\times{\mathbb R}_*^{n+1}$. With this action, $X_1$ is transitive under $G$. Let $x^0=(e_0, e_0)\in X_1$ where $e_0$ is the first standard unit vector in ${\mathbb R}^{n+1}$, i.e., $e_0={^t(}1, 0, \cdots, 0)$. Then the stabilizer of $x^0$ in $G$ is $H_1$. An elementary proof shows that $X_1\simeq G/H_1$. We also have $X\simeq G/H$ where $X=\{x\in{\rm M}_{n+1}(\mathbb R): {\rm rank}x={\rm tr}x=1\}$, here ${\rm M}_{n+1}(\mathbb R)$ is the space of all real $(n+1)\times(n+1)$ matrices. $G$ acts on ${\rm M}_{n+1}(\mathbb R)$ by conjugation (see [5])

$\begin{equation}g\cdot x=gxg^{-1}\ \ (g\in G, x\in{\rm M}_{n+1}(\mathbb R)).\end{equation} $

(2.2)

Let ${\mathfrak g}={\mathfrak{sl}}(n+1, {\mathbb R})$ be the Lie algebra of $G$. The Killing form of ${\mathfrak g}$ is $B(X, Y)=2(n+1){\rm tr}XY$ for $X$, $Y\in{\mathfrak g}$. The Killing form induces a measure on $X_1$. With this measure, the Casimir operator $\Omega$ of ${\mathfrak g}$ induces a second order differential operator on $X_1$. We call it the Laplace operator and denote it as $\square_1$.

For $\lambda\in{\mathbb R}$, set $\chi_{0}(t)=t^{\sqrt{-1}\lambda}$, $t\in{\mathbb R}_*$ be a continuous unitary character of ${\mathbb R}_*$. Define a character $\chi_{_\lambda}$ of $H$ as $\chi_{_\lambda}(h)=\chi_{0}(h_0)=h_0^{\sqrt{-1}\lambda}$ for

$h=\left(\begin{array}{cc}h_0& \\ &h_1 \end{array}\right)\in H.$

Let ${\mathcal D}(X_1)$ be the space of complex-valued $C^\infty$-functions on $X_1$ with compact support. The action of $G$ on $X_1$ induces a representation $U$ of $G$ in ${\mathcal D}(X_1)$:

$U(g)f(x)=f(g^{-1}x), \ \ \ \ g\in G, \ \ x\in X_1, \ \ f\in{\mathcal D}(X_1)$

and by inverse transposition a representation $U$ of $G$ in ${\mathcal D}^\prime(X_1)$.

We define

${\mathcal D}^\prime(X_1, \chi_{_\lambda})=\{T\in{\mathcal D}^\prime(X_1): U(h)T=\chi_{_\lambda}(h)^{-1}T, \ \ h\in H\}.$

Because $\chi_{_\lambda}=1$ on $H_1$, the above distributions $T$ can be viewed as the bi-$H_1$-invariant distributions on $G$ satisfying $U(h)T=\chi_{_\lambda}(h)^{-1}T, \ h\in H.$

If $\mu\in{\mathbb C}$, define

${\mathcal D}^\prime(X_1, \chi_{_\lambda}, \mu)=\{T\in{\mathcal D}^\prime(X_1, \chi_{_\lambda}): \square_1^\prime T=\mu T\}, $

where $\square_1^\prime$ is the transpose of the Laplace operator $\square_1$.

Definition 2.1 The $\chi_{_\lambda}$-spherical distributions $T$ on $X_1$ are the distributions on $G$ satisfying the following properties

$\bullet T$ is $H_1$-invariant,

$\bullet T(hx)=\chi_{_\lambda}(h)T(x)$, $h\in H$, $x\in X_1$,

$\bullet \square_1^\prime T=\mu T$ for some $\mu\in{\mathbb C}$.

As in [2], we define a mapping $Q_1: X_1\to{\mathbb R}$ by $Q_1(x, y)=x_0y_0$. We take the open subsets $X_1^0=\{(x, y)\in X_1: Q_1(x, y)<1\}$ and $X_1^1=\{(x, y)\in X_1: Q_1(x, y)>0\}$ of $X_1$. There is an averaging mapping ${\mathcal M}_1: f\mapsto{\mathcal M}_1f$ defined by

${\mathcal M}_1f(t)=\int_{X_1}f(x, y)\delta(Q_1(x, y)-t)d(x, y), $

where $\delta$ is the Dirac measure and $d(x, y)$ is a $G$-invariant measure on $X_1$. Define $\xi: X_1\to{\mathbb R}^2$ by $\xi(x, y)=(\xi_1(x, y), \xi_2(x, y))=(x_0, y_0)$. Then $\chi_{0}\circ\xi_1(x, y)=x_0^{\sqrt{-1}\lambda}$. Let ${\mathcal M}_{1, \sqrt{-1}\lambda}^\prime=(\chi_{0}\circ\xi_1)\cdot{\mathcal M}_1^\prime$ where ${\mathcal M}_1^\prime$ is the adjoint of ${\mathcal M}_1$. Then we have the main theorem of this paper.

Theorem 2.1 There is a second order differential operator $L_\lambda$ on ${\mathbb R}$ such that the following formula holds

$\begin{equation}\square_1^\prime\circ{\mathcal M}_{1, \sqrt{-1}\lambda}^\prime={\mathcal M}_{1, \sqrt{-1}\lambda}^\prime\circ L_\lambda, \end{equation}$

(2.3)

where

$\begin{equation}L_\lambda=4t(t-1){\displaystyle\frac{d^2}{dt^2}}+[4((n+1)t-1)+4\sqrt{-1}\lambda(t-1)]{\displaystyle\frac{d}{dt}}+ {\displaystyle\frac{2n}{n+1}}\sqrt{-1}\lambda(\sqrt{-1}\lambda+n+1).\end{equation}$

(2.4)

We take a basis of ${\mathfrak g}={\mathfrak{sl}}(n+1, {\mathbb R})$ as

$\left\{E_{11}-{\displaystyle\frac{1}{n}}\mathop \sum \limits_{i = 2}^{n + 1} E_{ii}, E_{jj}-E_{j+1, j+1} (2\le j\le n), E_{1s}, E_{s1} (2\le s\le n+1), E_{kl} (2\le k\ne l\le n+1)\right\}, $

where $E_{\alpha\beta}=(\delta_{\alpha\mu}\delta_{\beta\nu})_{\mu\nu}$ is as usual.

On $X_1$ we take the coordinates $\{x_0, y_0, x_1, y_1, \cdots, x_{n-1}, y_{n-1}, x_n\}$. Using (2.1), we follow the way in [6] to express $E_{\alpha\beta}$ as differential operators on $X_1$ in terms of the coordinates $\{x_0, y_0, x_1, y_1, \cdots, x_{n-1}, y_{n-1}, x_n\}$. The results are

${E_{11}} - \frac{1}{n}\sum\limits_{i = 2}^{n + 1} {{E_{ii}}} = {x_0}\frac{\partial }{{\partial {x_0}}} - {y_0}\frac{\partial }{{\partial {y_0}}} + \frac{1}{n}\left( { - \sum\limits_{i = 1}^n {{x_i}} \frac{\partial }{{\partial {x_i}}} + \sum\limits_{i = 1}^{n - 1} {{y_i}} \frac{\partial }{{\partial {y_i}}}} \right), $

(3.1)

$E_{1j}=x_{j-1}{\displaystyle\frac{\partial}{\partial x_0}}- y_0{\displaystyle\frac{\partial}{\partial y_{j-1}}}, \ \ \ \ 2\le j\le n, $

(3.2)

$E_{1, n+1}=x_n{\displaystyle\frac{\partial}{\partial x_0}}, $

(3.3)

$E_{j1}=x_0{\displaystyle\frac{\partial}{\partial x_{j-1}}}- y_{j-1}{\displaystyle\frac{\partial}{\partial y_{0}}}, \ \ \ \ 2\le j\le n, $

(3.4)

$E_{n+1, 1}=x_0{\displaystyle\frac{\partial}{\partial x_n}}- {\displaystyle\frac{1-x_0y_0-x_1y_1-\cdots-x_{n-1}y_{n-1}}{x_n}}{\displaystyle\frac{\partial}{\partial y_0}}.$

(3.5)

Following [1], let the function $F$ on $X_1$ be the form $F(x, y)=F(x_0, y_0)$. We calculate the action of the Laplace operator $\square_1$ or the Casimir operator $\Omega$ on such functions. Because $F$ depends on $x_0$, $y_0$ only, we take $\Omega$ as

$\begin{equation}2(n+1)\Omega={\displaystyle\frac{n}{n+1}}\left(E_{11}-{\displaystyle\frac{1}{n}}\mathop \sum \limits_{i = 2}^{n + 1} E_{ii}\right)^2 +\mathop \sum \limits_{k = 2}^{n + 1} (E_{1k}E_{k1}+E_{k1}E_{1k})+{\rm other\ terms}, \end{equation}$

(3.6)

where the `other terms' are the combinations of $E_{kl} (2\le k\ne l\le n+1)$. With the coordinates $\{x_0, y_0, x_1, y_1, \cdots, x_{n-1}, y_{n-1}, x_n\}$, using (3.1)-(3.5), we have

$\begin{array}{l} 2(n + 1)\Omega F({x_0}, {y_0}) = \left\{ {\frac{n}{{n + 1}}} \right.\left( {x_0^2\frac{{{\partial ^2}}}{{\partial x_0^2}} + y_0^2\frac{{{\partial ^2}}}{{\partial y_0^2}}} \right) + \left( {\frac{2}{{n + 1}}{x_0}{y_0} - 2} \right)\frac{{{\partial ^2}}}{{\partial {x_0}\partial {y_0}}}\\ \left. { + \frac{{n(n + 2)}}{{n + 1}}{x_0}\frac{\partial }{{\partial {x_0}}} + \frac{{n(n + 2)}}{{n + 1}}{y_0}\frac{\partial }{{\partial {y_0}}}} \right\}F({x_0}, {y_0}). \end{array}$

(3.7)

Now taking function $F(x_0, y_0)$ with the form $F(x_0, y_0)=x_0^{\sqrt{-1}\lambda}F_0(x_0y_0)$ and $t=x_0y_0$, we obtain

$\begin{equation}4(n+1)\Omega(x_0^{\sqrt{-1}\lambda}F_0(x_0y_0))=x_0^{\sqrt{-1}\lambda}L_\lambda F_0(x_0y_0)\end{equation}$

(3.8)

with

$\begin{equation}L_\lambda=4t(t-1){\displaystyle\frac{d^2}{dt^2}}+[4((n+1)t-1)+4\sqrt{-1}\lambda(t-1)]{\displaystyle\frac{d}{dt}}+ {\displaystyle\frac{2n}{n+1}}\sqrt{-1}\lambda(\sqrt{-1}\lambda+n+1).\end{equation}$

(3.9)

For $f\in{\mathcal D}(X_1)$, $T\in{\mathcal D}'({\mathbb R})$,

$\begin{eqnarray}&&\int_{X_1}\square_1[(T\circ Q_1)\cdot\xi_1^{\sqrt{-1}\lambda}](x, y)f(x, y)d(x, y)\nonumber\\ &=&\int_{X_1}(L_{\lambda}T)(Q_1(x, y))\cdot\xi_1^{\sqrt{-1}\lambda}(x, y)f(x, y)d(x, y) =\int_{\mathbb R}(L_{\lambda}T)(t)\int_{Q_1(x)=t}\xi_1^{\sqrt{-1}\lambda}(x)f(x)dxdt\nonumber\\ &=&\langle L_{\lambda}T, {\mathcal M}_1\xi_1^{\sqrt{-1}\lambda}f\rangle_{{\mathbb R}}=\langle {\mathcal M}_1^\prime L_{\lambda}T, \xi_1^{\sqrt{-1}\lambda}f\rangle_{X_1}=\langle \xi_1^{\sqrt{-1}\lambda}{\mathcal M}_1^\prime L_{\lambda}T, f\rangle_{X_1}, \end{eqnarray}$

(3.10)

$\begin{eqnarray}&&\int_{X_1}\square_1[(T\circ Q_1)\cdot\xi_1^{\sqrt{-1}\lambda}](x, y)f(x, y)d(x, y)\nonumber\\ &=&\int_{X_1}(T\circ Q_1)(x, y)\cdot\xi_1^{\sqrt{-1}\lambda}(x, y)({\square}_1f)(x, y)d(x, y)\nonumber\\ &=&\langle T, {\mathcal M}_1(\xi_1^{\sqrt{-1}\lambda}\square_1f)\rangle_{\mathbb R}=\langle{\mathcal M}_1^\prime T, \xi_1^{\sqrt{-1}\lambda}\square_1f\rangle_{X_1} =\langle\square_1{\mathcal M}_{1, \sqrt{-1}\lambda}^\prime T, f\rangle_{X_1}. \end{eqnarray}$

(3.11)

Comparing (3.10) and (3.11), we have $\square_1\circ{\mathcal M}_{1, \sqrt{-1}\lambda}^\prime={\mathcal M}_{1, \sqrt{-1}\lambda}^\prime\circ L_{\lambda}.$ This completes the proof of Theorem 2.1.