INTEGRAL GEOMETRY ON SL(2; ℝ)

Simon Gindikin

Abstract. We define a complex horospherical transform on the group SL(2; ℝ)

which corresponds to the Plancherel formula on it.

Subj.Class.: Lie Groups 1991 MSC: 22E46, 44A12 Keywords: Horospherical transform, symmetric space, Plancherel formula, oper-

ator κ.

Gelfand and Graev found that the computation of the Plancherel measure for complex semisimple Lie groups or noncompact Riemannian symmetric spaces is equivalent to a problem of geometrical analysis - the inversion of the horospherical transform [GGr,GGrV]. Unfortunately, this result does not admit a direct generalization on real semisimple Lie groups or, more generally, on semisimple pseudo-Riemannian symmetric spaces: the horospherical transform for them has a kernel, corresponding to the discrete series of representations. Gelfand several times suggested the problem to find a version of the integral geometry, corresponding to the harmonic analysis on real groups, starting with SL(2; ℝ). We develop in this paper such integral geometry on SL(2; ℝ). The starting idea is very simple. The desired integral geometry uses complex horospheres in the SL(2; ℂ)\SL(2; ℝ) instead of real horospheres in SL(2; ℝ) and Cauchy kernels instead of δ-functions. This “complex” horospherical transform already has no kernel and gives the ability to write the Plancherel formula, including discrete series. Let us remark that a complex language is useful already for the real Radon transform [Gi1]. Connections of representations of SL(2; ℝ) with a complex geometry of SL(2; ℂ)\SL(2; ℝ) was considered in [GGi] and our considerations here in a sense develop the ideology of this paper (cf.also [Gi2]). I hope that these results give a good chance to develop the integral geometry on pseudo-Riemannian symmetric spaces, including real semisimple Lie groups, which corresponds to the harmonic analysis on them.

I thank Sasha Goncharov for many discussions.

Complex geometry. Essentially we do not use the group structure of G_{ℂ} = SL(2; ℂ) of 2 × 2-matrices:

g = ( \begin{matrix}α & β \\ γ & δ \end{matrix} ) , det g = 1

Typeset by AMS-TEX

Supported in part by NSF grant DMS97-06836. The author thanks Max-Planck-Institut für Mathematik at Bonn for hospitality.

and consider it as the hyperboloid in ℂ^{4}:

(1)  □g = αδ − βγ = 1.

It is a (pseudo-Riemannian) symmetric space G_{ℂ} × G_{ℂ}/G_{ℂ} (relative to left and right multiplications). The group G_{ℂ} × G_{ℂ} is locally isomorphic to SO(4; ℂ).

Let the bilinear form g_{1} · g_{2} in ℂ^{4} be the polarization of the quadratic form □g such that we can can rewrite (1) as

g · g = 1.

If det g_{2} = 1 then

g_{1} · g_{2} = \frac{1}{2} tr(g_{1}g_{2}^{−1}).

Often in our computations we will transform the hyperboloid (1) in the hyperboloid

(1′)  □z = z_{1}^{2} + z_{2}^{2} − z_{3}^{2} − z_{4}^{2} = 1,

replace correspondingly the form z·w and consider the action of the group SO(2; 2), locally isomorphic to G_{ℂ} × G_{ℂ}.

Basic objects of integral geometry are horospheres. There are two alternative types of horospheres which can be put in the base of the integral geometry on SL(2; ℂ): one-dimensional horocycles and two-dimensional horospheres. Let us start from the description of horospheres. They are parameterized by the points of the cone

(2)  Ξ = {ζ; □ζ = ζ · ζ = αδ − βγ = 0, ζ ≠ 0}.

We have

(3)  Ξ = G_{ℂ} × G_{ℂ}/C,  ζ ↦ g_{1}^{⊤}ζg_{2},

where C is the direct product of Borelian (triangle) subgroups B with the joint Cartan (diagonal) subgroups.

For ζ ∈ Ξ we define the horosphere Ω(ζ) (of the dimension 2) as the section of the hyperboloid G (1) by the hyperplane

(4)  ζ · g = 1, □ζ = 0.

These sections are 2-dimensional hyperbolic paraboloids (of course in the complex picture all quadrics are equivalent) .

Usually we will work with an extended family of horospheres Ξ̃. It includes also “limit” horospheres Ω(ζ, 0) - sections of G by hyperplanes

ζ · g = 0, □ζ = 0.

These sections are cylinders. We can represent hyperplanes from the extended family Ξ̃ as Ω(ζ, p):

(5)  ζ · g = p, □ζ = 0;

(ζ, p) are homogeneous coordinates in Ξ̃. Of course, the extended space of horospheres Ξ̃ is not homogeneous relative to the SO(2, 2)-action.

Let us define another (projective) set of coordinates on the manifold Ξ of horospheres. Let U, V ∈ ℂ^{2} be realized as row-vectors. Then elements of Ξ can be present as

ζ = U^{⊤}V

and we can parameterize horospheres by triplets (U, V, p) such that

(λU, μV, λμp) ∼ (U, V, p), λ, μ ∈ ℝ

and the action ζ ↦ g_{1}^{⊤}ζg_{2} corresponds to the action U ↦ Ug_{1}, V ↦ V g_{2}. We will use affine coordinates for ζ in the coordinate chart

(6)  \begin{matrix}V_{1} = v, V_{2} = 1, U_{1} = λu, U_{2} = λ, \\ ζ(λ|u, v) = ( \begin{matrix}λuv & λu \\ λv & λ \end{matrix} ) . \end{matrix}

Correspondingly we will write Ω(ζ, p) = Ω(λ|u, v; p). In these coordinates G_{ℂ} × G_{ℂ} acts in the following way;

(7)  \begin{matrix}g_{1}^{⊤}(λ|u, v)g_{2} = (λ̃|ũ, ṽ) \\ ũ = g_{1}(u), ṽ = g_{2}(v), g(u) = \frac{αu + γ}{βu + δ} , \\ λ̃ = λ(β_{1}u + δ_{1})(β_{2}v + δ_{2}). \end{matrix}

Horocycles are exactly the linear generators on the hyperboloid G (3-parametric family). They have the form ω(ζ, a):

(8)  g = a + tζ, □ζ = 0, ζ · a = 0.

The point a here is defined up to any multiple of ζ. Let Ξ′ be a set of horocycles; as a homogeneous space Ξ′ = G_{ℂ} × G_{ℂ}/C, where C is the product of two copies of Borel subgroups B with the joint Cartan subgroups.

The family Ξ′ of horocycles includes the unipotent subgroup Z = ( \begin{matrix}1 & t \\ 0 & 1 \end{matrix} ) and

coincides with all its translations by elements G_{ℂ} × G_{ℂ}:

(9)  ω[g_{1}, g_{2}] = g_{1}^{−1}Zg_{2}.

It is possible to define a projective parameterization of Ξ′ similar to the parameterization of Ξ, but we give here only the affine parameterization, induced by g_{1}, g_{2} ∈ B^{⊤} at (9): the horocycle ω[λ|u, v] consists of matrices

(10)  ( \begin{matrix}λ^{−1} + vt & t \\ −λ^{−1}u + λv − uvt & λ − ut \end{matrix} ) .

The parameters (λ|u, v) here transform on formulas (7). We have the connections between two parameterizations:

ω[λ|u, v] = ω(ζ, a), where ζ = ( \begin{matrix}v & 1 \\ −uv & −u \end{matrix} ) , a = ( \begin{matrix}λ^{−1} & 0 \\ −λ^{−1}u + λv & λ \end{matrix} ) .

Between horospheres and horocycles there is a simple connection of incidence. Firstly, a limit horosphere Ω(ζ, 0) is the union of horocycles ω(ζ, a) with the same ζ (let us remind that limit horospheres are cylinders). Limit horospheres for nonproportional ζ do not intersect.

The horosphere Ω(ζ, p) contains a horocycle ω(ζ̃, a) if and only if

(11)  ζ · ζ̃ = 0, ζ · a = p.

For a fixed ζ the first equation gives the section of the cone Ξ which is a pair of 2-subspaces intersecting on the line {tζ}. If we are to consider ζ̃ as homogeneous coordinates, we have 2 lines, intersecting at ζ. If p ≠ 0 and ζ̃, ζ are nonproportional, we can find an unique a ∈ G, up to a multiple of ζ̃, from (11) and the condition ζ̃·a = 0. Therefore the family of horocycles ω(ζ̃, a) on the horosphere Ω(ζ, p), p ≠ 0, is parameterized by ζ̃ on the pair of affine lines, which are obtained from the intersection of Ξ by the hyperplane in (11) and by the removal of the point ζ. Let us recall that horospheres Ω(ζ, p), p ≠ 0, are hyperbolic paraboloids and they have 2 families of linear generators. Correspondingly, horospheres Ω(ζ, p), containing a horocycle ω(ζ̃, a), are parameterized by points of the pair of (projective) lines, intersecting in ζ̃ and defined by (11); the point ζ̃ corresponds to the limit horosphere Ω(ζ̃, 0). In such a way we have the duality between horospheres and horocycles.

In the integral geometry on SL(2; ℂ) we consider the horospherical transform: the integration either along horospheres or (more often) along horocycles ([GGrV]). These two versions of the horospherical transform have a very simple connection: in a sense 2-dimensional transform is the composition of the 1-dimensional one and the usual 2-dimensional (complex) Radon transform. For both transforms there are explicit inversion formulas which can be transformed in the Plancherel formula for the group SL(2; ℂ), using the Mellin transform.

Real geometry. Let us consider the real group (hyperboloid) G_{ℝ} = SL(2; ℝ) and the action of G_{ℝ} × G_{ℝ} on G_{ℂ}. Let g = x + iy. We have

(12)  □x − □y = x · x − y · y = 1, x · y = 0.

On G_{ℂ}\G_{ℝ} we have a different kind of orbits which we will characterize by some canonical representatives. There are five domains, corresponding to different types of orbits for this action: the domains G_{±} are the connect components of the set {□y > 0}; the domain G_{0} is defined by the condition {□y < 0, □x > 0} and the domains G_{1}, G_{2} are the connect components of the set {□y < 0, □x < 0}.We give below descriptions of these domains on the language of canonical representatives of orbits.

Elements of G_{±} can be transform to the form:

(13)  ( \begin{matrix}λ & iμ \\ −iμ & λ \end{matrix} ) , λ^{2} − μ^{2} = 1, λ > 0, μ ≷ 0

correspondingly. Points of G_{0} can be transform to the canonical form:

(14)  ( \begin{matrix}λ + iμ & 0 \\ 0 & λ − iμ \end{matrix} ) , λ^{2} + μ^{2} = 1

and points of G_{1}, G_{2} to the form:

(15)  ( \begin{matrix}λ & iμ \\ iμ & −λ \end{matrix} ) , λ^{2} − μ^{2} = −1, λ > 0, μ ≷ 0.

First three domains are tubes with G_{ℝ} as the edge; G_{±} are Stein manifolds (in [GGi] was shown that they are equivalent Zarisky open parts in Siegel half-planes); G_{0} is 1-pseudoconcave. The boundaries of the domains G_{1}, G_{2} do not intersect G_{ℝ} and as a result these domains are not essential for our considerations.

In the coordinates (1′) we can take

(13′)  z = (λ, iμ, 0, 0), λ^{2} − μ^{2} = 1, λ > 0, μ ≷ 0

as representatives of G_{±},

(14′)  z = (λ, 0, iμ, 0), λ^{2} + μ^{2} = 1

as representatives of G_{0} and

(15′)  z = (0, 0, λ, iμ), λ^{2} − μ^{2} = −1, λ > 0, μ ≷ 0

as representatives of G_{1}, G_{2}.

Complex horospheres in the real picture. Our principal geometrical problem is to find complex horospheres without real points (which do not intersect G_{ℝ}). We consider complex horospheres (including limit ones) Ω(ζ, p) (5). It is sufficient to consider p = 0, 1. Let ζ = ξ + iη. Then

(16)  □ξ = □η = 0, ξ · η = 0.

Proposition. Complex horosphere Ω(ζ, p) has no real points in three situations: (i) □ξ − |p|^{2} > 0; (ii) □ξ = 0, η = 0, ℑp ≠ 0; (iii) □ξ = □η = 0, p = 0, ξ ≠ rη, r ≠ 0.

Proof. We will use the representation (1′) and we are interesting when (5) has no solutions z = x, □x = 1, ℑz = 0.

Let □ξ > 0, p = 1. Then we can transform ξ to the form ξ = (λ, 0, 0, 0), λ > 0, η to the form η = (0, ±λ, 0, 0). It must be ξ · x = 1, η · x = 0, therefore x_{1} = λ^{−1}, x_{2} = 0, x_{3}^{2} + x_{4}^{2} = −1 + λ^{−2}. So we have a real solution if and only if λ > 1 what corresponds in this case to (i).

If □ξ > 0, p = 0, then for similar reasons we take ζ = (λ, ±iλ, 0, 0) and it must be that x_{1} = x_{2} = 0 and we have no solutions since we have the condition x_{3}^{2} + x_{4}^{2} = −1.

If □ξ < 0, then we can transform ζ to ζ = (0, 0, λ, iλ) and for both p = 0, 1 we can find real solutions x.

Let □ξ = □η = 0. If η = 0, ℑp = 0, we have a real horosphere on G_{ℝ}. If η = 0, ℑp ≠ 0, the horospheres apparently have no real points. If η ≠ 0, we can suppose, that η = (a, 0, a, 0), a ≠ 0, and then on (16) we have ξ = (b, c, b, ±c). For a solution x with p = 1 we have x_{1} = x_{3}, c(x_{2} ∓ x_{4}) = 1 and solutions exist if and only if c ≠ 0. If c = 0 we can to change parameters such that η = 0. So we obtain a condition equivalent to (ii).

If p = 0 and c ≠ 0, then again x_{1} = x_{3} and either x_{2} = x_{4} or x_{2} = −x_{4}. In both cases we have □x = 0 and we obtain the contradiction with □x = 1. So these horospheres have no real points and we obtain the condition (iii).

The set of parameters of horospheres satisfying condition (i) of Proposition has two connected components Ξ_{±}. Let us describe them in the coordinates (6). Let p = 1 and then Ξ_{±} will be the sets of ζ ∈ Ξ, equivalent to (λ, ±iλ, 0, 0), λ > 1, correspondingly. In matrix coordinates (1) such points have representatives

(17)  ζ(λ| ± i, ∓i) = ( \begin{matrix}λ & ±iλ \\ ∓iλ & λ \end{matrix} ) , λ ∈ ℝ, λ > 1

and we need to describe the union of corresponding orbits of G_{ℝ} × G_{ℝ} (7). Let us start of Ξ_{+}. Points u = i, v = −i have the joint isotropy subgroup of orthogonal matrices g. As the result of its action we have ζ(λ̃|i, −i), λ̃ ∈ ℂ, |λ̃| > 1. The translation which transforms (i, −i) in (u, v), ℑu > 0, ℑv < 0, transforms λ̃ in λ with |λ| = |λ̃|(−ℑuℑv)^{−1/2}. Then we obtain ζ(λ|u, v) with (−ℑuℑv)|λ|^{2} > 1, ℑu > 0, ℑv < 0. Correspondingly, for Ξ_{−} we have ℑu < 0, ℑv > 0. For p ≠ 1 we obtain the description of Ξ_{±}:

(18)  (−ℑuℑv)|λ|^{2} > |p|^{2}, ℑ(±u) > 0, ℑ(∓v) > 0.

Apparently, this description is correct also for p = 0.

The set of parameters of horospheres satisfying to (ii) we will denote through Ξ_{0}: we can take for them real ζ = ξ and nonreal p. Horospheres, satisfying to (iii), will not participate in our results and we do not introduce a notation for this set (it lies on the boundary of Ξ_{±}).

In the relation to horocycles in the real picture we will consider only horocycles ω(ξ, a), where ξ is real. We will distinguish three types of such horocycles. Let us remark that a in their parameterization are unique up to the addition of an multiple of ξ: ã = a + cξ. We will say that the horocycle has the real (imaginary) type if it is possible to choose real (imaginary) a (such that if ℜa (correspondingly ℑa) is proportional to ξ). In other cases we will say that we have a generic horocycle.

Connection between horospheres, horocycles and domains in the real picture. Let G^{∓} be the supplements to the closure of G_{±} correspondingly.

Proposition. (i)The domains G^{±} coincide with the unions of horospheres Ω(ζ, p), (ζ, p) ∈ Ξ_{±}.

(ii) The domain G_{0} coincides with the union of generic horocycles ω(ξ, a). The union of such horocycles of real (imaginary) type coincides with the part of the boundary of G_{0} where □y = 0, □x = 1 (correspondingly □x = 0, □y = −1).

Proof. (i)We will work in the representation (1′). The basic thing which we need to proof is that the horospheres Ω(ζ, p), (ζ, p) ∈ Ξ_{−}, do not pass through points of G_{+}. We can reformulate it the following way. Let Π be the set of such ζ, □ζ = 0, that □ξ = 1. They can be transformed to one of 2 canonical forms (1, ±i, 0, 0). Correspondingly, the set Π has 2 components Π_{±}. We need to prove that

(19)  |ζ · z| > 1 for ζ ∈ Π_{−}, z ∈ G_{+}.

By virtue of invariance, it is enough to prove for ζ = (1, −i, 0, 0):

(19′)  |z_{1} − iz_{2}| > 1 for z ∈ G_{+}.

We have |z_{1} −iz_{2}|^{2} = (x_{1} +y_{2})^{2} +(y_{1} −x_{2})^{2} = (x_{1}^{2} +x_{2}^{2})+(y_{1}^{2} +y_{2}^{2})+(x_{1}y_{2} −x_{2}y_{1}). As far as z ∈ G_{+}, then □y > 0 and (y_{1}^{2} + y_{2}^{2}) > 0; □x > 1 and (x_{1}^{2} + x_{2}^{2}) > 1. Therefore it is sufficient to check that

(19′′)  (x_{1}y_{2} − x_{2}y_{1}) > 0  on G_{+}.

Let us verify that this expression can not be zero on G_{+}. If x̃ = (−x_{2}, x_{1}, 0, 0) (we have in (19′′) x̃ · y), then □x̃ > 1, since □x > 1; x̃ · x = 0 . Suppose that x̃ · y = 0. Together with the condition x · y = 0 (on the definition) it gives y = 0. Indeed, using the invariance, we can suppose x = (a, 0, 0, 0), x̃ = (0, b, 0, 0), a ≠ 0, b ≠ 0, and it must be y_{1} = y_{2} = 0 and which is a contradiction to the condition □y > 0. Thus x̃ · y conserves a sign on G_{+} and therefore positive since it is positive for canonical elements (13′) in G_{+}.

It remains to check, that for almost every element of G^{−} there is a horosphere with parameters in Ξ_{−} , which passes through it. It is sufficient to consider the canonical representatives of G_{−}, G_{0}, G_{1}, G_{2} of (13′) - (15′) and to find such ζ ∈ Π_{−} that |ζ · z| < 1. In all cases ζ = (1, −i, 0, 0) satisfies to this condition.

Remark. Horospheres Ω(ζ, 1), ζ ∈ Π, are supporting ones for the real hyperboloid G_{ℝ}. Moreover, they intersect the hyperboloid only in one point x = ξ. It is sufficient to consider ζ = (1, ±i, 0, 0). Then for the intersection points must be true that x_{1} = 1, x_{2} = 0 and on G_{ℝ} there is only one point – (1,0,0,0) – with this condition. Supporting horospheres in each point of G_{ℝ} are parameterized by points of a 2-dimensional hyperboloid of 2 sheets. They will play an important role in our considerations.

(ii) The basic fact is that there is no points z with □(ℑz) > 0 on limit horospheres Ω(ξ, 0) with a real ξ. Indeed, let ξ · z = 0, z = x + iy, □y > 0. Using the invariance, we can consider only ξ = (1, 0, 1, 0). Then z_{1} = z_{3} and z_{2}^{2} − z_{4}^{2} = 1. So

(20)  x_{2}^{2} − x_{4}^{2} − y_{2}^{2} + y_{4}^{2} = 1, x_{2}y_{2} − x_{4}y_{4} = 0

Let □y > 0. By the action of a transform, conserving ξ, we can obtain y with y_{4} = 0, y_{2} ≠ 0. Then x_{2} = 0 and we obtain the nonrealizable condition:−x_{4}^{2}−y_{2}^{2} = 1.

If □y = 0, then x_{2}^{2} − x_{4}^{2} = 1 and we can transform x to the case x_{4} = 0, x_{2} = ±1 in (20). Then y_{2} = 0, y_{4} = 0, y = cξ and such points lie on horocycles ω(ξ, a) with real a. On the other side, ℑz = cξ on horocycles of real type and hence □y = 0. Each point z with □y = 0 can be transform in the point (1, iλ, iλ, 0), which lies on the horocycle ω(ξ, a) with a = (1, 0, 0, 0), ξ = (0, 1, 1, 0) of real type. Such a way we connect a part of the boundary of G_{0}, where □y = 0, with horocycles of real type.

For other types of horocycles □y < 0. Then in the same notation as in (20) we can suppose that y_{4} ≠ 0, y_{2} = 0 and as a result x_{4} = 0 and □x ≥ 0.

For our representatives the condition □x = 0 means that x_{2} = x_{4} = 0 and we have a horocycle of imaginary type since ℜx = cξ. Again for points of any such a horocycle we have ℜz = cξ, □x = 0. Points with conditions □x = 0, □y = −1 can be transform in points (0, λ, λ, i) which lie on horocycles of imaginary type with ξ = (0, 1, 1, 0), a = (0, 0, 0, i).

Finally, on generic horocycles we have □x > 0, □y < 0 hence they belong to the domain G_{0}. Since there is a generic horocycle ω(ξ, a) passing through the canonical point z = (0, λ, iμ, 0) (ξ = (1, 0, 0, 1), a = z), for any point of G_{0} there is a passing generic horocycle. The proof is finished.

Complex horospherical transform. Let us start our analytic constructions from some notations. Let

dx = dx_{1} ∧ dx_{2} ∧ dx_{3} ∧ dx_{4}

and

(21)  μ(x, dx) = d(□x)⌋dx = \frac{dx_{2} ∧ dx_{3} ∧ dx_{4}}{2x_{1}}

be the invariant measure on the hyperboloid G_{ℝ} (in the form (1′)). In our formulas we denote through [a_{1}, a_{2}, …, a_{k}] the determinant with the columns a_{1}, …, a_{k}. Some of them can be columns of 1-forms and we use the cup-product for the computing of the determinant. As a result a determinant with identical columns of 1-forms can be different from zero. The important example of such a determinant is the form Leray

(22)  L(ξ, dξ) = [ξ, dξ, …, dξ], ξ ∈ ℝ^{k},

where the column dξ repeats (k − 1) times. This form plays an important role in the projective analysis.

Let H be the set of horospheres Ω(ζ, p) without real points, H_{±} be components of the set horospheres satisfying to the condition (i) of Proposition and H_{0} corresponds to the condition (ii). For p = 1 we will identify these sets with Ξ_{±}, Ξ_{0} correspondingly. For f(x) ∈ C_{0}^{∞}(G_{ℝ}) we define the (complex) horospherical transform as

(23)  f̂(ζ, p) = ∫_{G_{ℝ}} \frac{f(x)}{ζ · x − p}μ(dx), (ζ, p) ∈ H.

Since Ω(ζ, p) has no real points, this transform is well defined. Let f̂_{±}, f̂_{0} be its restrictions on H_{±}, H_{0} correspondingly. The functions f̂_{±} are holomorphic and f̂_{0} is a CR-function.

Our principal result is the possibility to reconstruct f through the horospherical transform f̂. We will give an explicit formula for the reconstruction of f through boundary values of f̂. For each x ∈ G_{ℝ} we consider the sets Γ_{±}(x) of horospheres Ω(ζ, 1), lying on the boundaries of H_{±}(Ξ_{±}) and passing through x. As we saw for them

(24)  ζ = x + iη, x · η = 0, □η = 1.

We use η as parameters and the conditions (24) defines a 2-dimensional hyperboloid of two sheets, giving Γ_{±}(x). If x = (1, 0, 0, 0), then (24) means η_{1} = 0 and Γ_{±}(x) are the sheets

(25)  (η_{2})^{2} − (η_{3})^{2} − (η_{4})^{2} = 1, η_{2} ≷ 0.

Let us denote L_{x}(η̃, dη̃) the Leray form (22) on Γ_{±}(x) on coordinates in which □ has the form (25) (for x = (1, 0, 0, 0) we take η̃ = (η_{2}, η_{3}, η_{4})).

Relative to the taking of boundary values, we remark that the horospheres Ω(x+ iη, 1 + ε), ε > 0, lies in H_{±} and we take the boundary values as ε → 0 which exist as distributions since we have the singularity only in the point x:

(26)  f̂_{±}(x + iη, 1) = lim_{ε→0} f̂_{±}(x + iη, 1 + ε).

Correspondingly, we take as Γ_{0}(x) the set of (real) horospheres Ω(ξ, 1), x · ξ = 1. Let ξ = x + ξ̃, x · ξ̃ = 0. Then □ξ̃ = −1 and we have a hyperboloid of one sheet. If x = (1, 0, 0, 0), then ξ̃_{1} = 0 and we have the hyperboloid of one sheet

(27)  (ξ̃_{2})^{2} − (ξ̃_{3})^{2} − (ξ̃_{4})^{2} = −1.

We define L_{x}(ξ̃, dξ̃) as above. Let f̂_{0}(ξ, 1) be the boundary values in the sense of distributions:

(28)  f̂_{0}(x + ξ̃, 1) = lim_{ε→0}(f̂_{0}(x + ξ̃, 1 + iε), x · ξ̃ = 0, □(ξ̃) = −1.

We take here the limit of f̂ for horospheres of H_{0}.

The horospherical transform f̂_{0}(ξ, 1) has a natural connection with the real horospherical transform

(29)  Rf(ξ, p) = ∫_{G_{ℝ}} f(x)δ(ξ · x − p)μ(x, dx), p ∈ ℝ.

Namely,

f̂_{0}(ξ, 1 + iε) = ∫_{−∞}^{∞} \frac{Rf(ξ, q)}{q − 1 − iε}dq.

Now we can formulate the principal result.

Main Theorem. For f(x) ∈ C_{0}^{∞}(G_{ℝ}) there exists an inversion formula

(30)  \begin{matrix}f(x) = f_{+}(x) + f_{−}(x) + f_{0}(x), x ∈ G_{ℝ} \\ f_{±}(x) = \frac{1}{4π^{3}} ∫_{Γ_{±}(x)} Lf̂_{±}(x + iη, 1)L(η̃, dη̃), \\ f_{0}(x) = \frac{1}{4(πi)^{3}} ∫_{Γ_{0}(x)} Lf̂_{0}(x + ξ̃, 1)L(ξ̃, dξ̃), \\ L = \frac{∂}{∂p} + p\frac{∂^{2}}{∂p^{2}} . \end{matrix}

The proof of this theorem uses a version [Gi3] of the central construction of integral geometry - the operator κ of Gelfand-Graev-Shapiro [GGrSh].

Operator κ (the proof of Main Theorem). The operator κ acts from functions on G_{ℝ} to differential 2-forms on G_{ℝ} × ℂ_{ζ}^{4}:

(31)  κf[x] = \frac{f(u)}{(ζ · (u − x))^{3}} μ(u, du) ∧ [u + x, ζ, dζ, dζ],  □x = □u = 1,

where x is a fixed point of G_{ℝ} and we write the coordinates of u + x as a column in the determinant. It is a general construction for arbitrary quadrics [Gi1, Gi3]. It plays a role of the decomposition of δ-function on plane waves for the case of quadrics. The basic result is

Proposition. The form κf[x] is closed.

The proof is a direct consequence of the differentiation formula

(32)  d[a(ξ), ξ, dξ, …, dξ] = (∑ ξ_{j}∂a^{j}/∂ξ_{j})L(ξ, dξ),

where a(ξ) is a column of homogeneous functions a^{j}(ξ) of the degree −k. The proof of this formula is a direct calculation involving the Euler formula for homogeneous functions (cf. [Gi3]).Relative to u we have the form of the maximal degree, so we need only to care about the differentiation on ζ. We include the factor (ξ·(u−x))^{−3} in the first column and after the application of (32) we obtain the factor □u−□x = 0.

The form κ is the basic tool to obtain new inversion formulas from simplest ones. We integrate the form κf[x] on G_{ℝ} × Γ, where Γ is a cycle in ζ-space. We show that for a cycle Γ this integral coincides with the Radon inversion formula and give cf(x). Then we deformate this cycle in the cycle of horospheres and through the virtue of the closeness of κ it gives us the inversion of the horospherical transform.It is essential that we realize this deformation in complex parameters.

To be more exact, we extend the set H of horospheres Ω(ζ, p) without real points up to set H̃ of all sections L(ζ, p) of G by hyperplanes

ζ · z = p

without real points and also consider sections, corresponding to boundary parameters (ζ, p) ∈ ∂H̃. For (ζ, p) ∈ H̃ we define f̂(ζ, p) by the formula (23) and then take the boundary values on holomorphic parameters. Thus all real pairs (ξ, p) correspond to “boundary” sections and we define f̂ for them as in (28).

Let us put in (31) ζ · x = p and integrate it in the beginning on u ∈ G_{ℝ}. This integral has a sense if (ζ, p) ∈ H̃ and gives a 2-form F (p; ζ, dζ), which can be express through f̂(ζ, p) by a differential operator of 2-nd order, which can be write down explicitly [Gi1, Gi3](but we do not need this an expression in the general case).Important that this form also is closed.

Then we substitute p = ζ · x and integrate the result on different cycles Γ in ζvariables.We can not in this situation to find cycles with the condition (ζ, ζ ·x) ∈ H̃, but we can find cycles with the condition (ζ, ζ · x) ∈ ∂H̃.

Firstly, we can take cycles with real ζ = ξ and f̂(ξ, p + i0).Let us take as the cycle Γ^{0}(x) any cycle homological to the unit sphere in the hyperplane {ξ; x·ξ = 0} (the integrals for all such cycles will coincide in view of the closeness of κ). Then

(33)  ∫_{G_{ℝ}×Γ^{0}(x)} κf = 4(πi)^{3}f(x).

It is a simple consequence of the usual Radon inversion formula in the projective form [GGiGr,Gi1].Namely if x = (1, 0, 0, 0), then on Γ^{0}(x) we have ξ_{1} = 0 and let ξ = (0, ξ̃), u = (u_{1}, ũ). Hence on Γ^{0}(x) the form κf coincides with

\frac{(u_{1} + 1)f(u)}{(ξ̃· ũ − i0)^{3}}L(ξ̃, dξ̃).

Then (33) for even f coincides with the projective Radon inversion formula (restricted on the hyperboloid) and for odd f with the same formula for the u_{1}f(u).

Let us deformate this cycle Γ^{0}(x) in the cycle of horospheres Γ^{1}(x) inside H̃ ⋃ ∂H̃ and through the virtue of closeness of κ we will obtain the inversion formula for the horospherical transform. We will describe this deformation for x = (1, 0, 0, 0). Let ξ = (0, ξ̃ ∈ Γ^{0}(x) and

q(ξ̃) = \sqrt{(−□ξ̃)},

where in the real case we take the positive evaluation of the root and in the imaginary case we take the evaluation of the form ai, a > 0.

Let us

(34)  ζ(ε) = (εq(ξ̃), ξ̃), p(ε) = εq(ξ̃), 0 ≤ ε ≤ 1,

and let Γ^{ε}(x) be the cycle of the sections L(ζ(ε), p(ε).

All these sections lie in ∂H̃.Indeed we need verify it only for the imaginary q. Then, for ε ≠ 0, L has only one intersection point x = (1, 0, 0, 0) with G_{ℝ}.The proof is the same as above, when we consider the support planes Π_{±}. If we replace p(ε) on p(ε)+iδ, δ > 0, we obtain the section without real points.Such a way the integral of κf on any cycle Γ^{ε}(x) has the same value. For ε = 1 we have the cycle Γ^{1}(x) of horospheres: □(ζ(0)) = 0. It decomposes on three parts: Γ_{0}(x) corresponds to ξ̃ with □(ξ) < 0; Γ_{±} correspond to 2 components of the set {□(ξ̃) > 0. If q(ξ̃) ≠ 0 we can transform pairs (ζ(1), p(1)) to the form (ζ, 1).

The last step is a transformation of the determinant in κ. We have ζ_{1} = 1.Replace the first row by the combination of all rows with the coefficients ζ_{j}. Then in the first row only the first element will be different of zero and equal ζ · u + 1. The whole determinant will be equal to (ζ ·u+1)L(ζ.dζ) and we can see directly that the factor in κf before L(ζ̃, dζ̃) is exactly the result of the application of the differential operator L to f̂ and it gives (30).

Spherical Fourier transform (discrete series). The connection with representations goes through the spherical Fourier transform. As we already mentioned the domains Σ_{±} are invariant (but not homogeneous) relative to the group G_{ℝ} × G_{ℝ}. It is very important that on these domains there is also a commutating action of the circle 𝕋. In the coordinates (λ, u, v) (7) it is the multiplication of λ on exp(iρ).

Let us consider f̂_{±}(ζ) as f̂_{±}(λ|u, v) which are holomorphic in the domains Ξ_{±} (18):

|λ|^{2} > \frac{1}{−ℑuℑv} , ℑ(±u) > 0, ℑ(∓v) > 0.

Let us decompose them in the Taylor series in these domains on λ:

(35)  f̂_{±}(λ|u, v) = ∑_{k>0} f̃_{±}(k|u, v)λ^{−k−1}, |λ| > (−ℑuℑv)^{−1/2}.

Let us call functions f̃_{±}(k|u, v) which are holomorphic in the products of two halfplanes by the discrete spherical Fourier transforms of f. As follows from (7) the group G_{ℝ} × G_{ℝ} acts on them in the following way:

(36)  \begin{matrix}T_{(g_{1},g_{2})}^{k}f̃_{±}(k|u, v) = f̃_{±}(k|g_{1}(u), g_{2}(v))(β_{1}u + δ_{1})^{−k−1}(β_{2}v + δ_{2})^{−k−1}, \\ ℑ(±u) > 0, ℑ(∓v) > 0. \end{matrix}

We see that the group G_{ℝ} × G_{ℝ} acts on the discrete spherical Fourier transform as the corresponding discrete (holomorphic and antiholomorphic) series of representations of this group.

Now we will rewrite the inversion formula (30) for f_{±} through the discrete spherical Fourier transform. We need to start from the definition of the action of the operator L (30). Let us pass from f̂_{±}(λ|u, v) to f̂_{±}(λ|u, v; p). Using the homogeneity of f̂, we need to replace in the series (35) λ^{−k−1} on λ^{−k−1}p^{k}, to apply the operator L (on p) and put p = 1. We obtain

(37)  Lf̂_{±}(λ|u, v) = ∑_{k>0} k^{2}f̃_{±}(k|u, v)λ^{−k−1}, |λ| > (−ℑuℑv)^{−1/2}.

So the operator L corresponds in the spherical Fourier transform to the multiplication on k^{2}.

The next step is to investigate the cycles Γ_{±}(x) in (30) in the parameters (λ, u, v). Let us work in the group coordinates (1) and let x = e - the unit element. Then Γ_{±}(e) consists of matrices

( \begin{matrix}1 + iη_{1} & iη_{2} \\ iη_{3} & 1 − iη_{1} \end{matrix} ) , −η_{1}^{2} − η_{2}η_{3} = 1, η_{2} ≶ 0.

We put here on the matrix η the conditions of the orthogonality to e (the trace is equal to 0) and we wrote down that the determinant is equal 1. So we have the sheets of the two-sheeted hyperboloid. We have on Γ_{±}(e), according to (6),

λ = 1 − iη_{1}, u = \frac{iη_{2}}{1 + (iη_{1})^{2}} , v = \frac{iη_{3}}{1 + (iη_{1})^{2}} .

As the result

(38)  v = −\frac{1}{\overline{u}},  λ^{−1} = \frac{1}{2}(uv + 1) = \frac{−u + \overline{u}}{2\overline{u}}.

It corresponds to the parameterization of one sheet of the hyperboloid by points of the upper (lower) half-plane Thus we need to substitute these v and λ in (37) and for summation of the series, since we have a point on the boundary of the domain of convergence (|λ| = (−ℑuℑv)^{−1/2}), we need for the regularization to include factors ρ^{−k−1}, ρ > 1 and take the limit as ρ → 1.We need also to express L(η̃, dη̃) through the parameters u (apparently it must be invariant relative to SL(2; ℝ)). Finally,

(39)  f_{±}(x) = \frac{2}{π^{3}} ∫_{ℑu≶0} (∑_{k>0} k^{2}f̃_{±} (k|u, −\frac{1}{\overline{u}}) (\frac{−u + \overline{u}}{2\overline{u}})^{−k−1}) \frac{du ∧ d\overline{u}}{|u|^{2}}.

Spherical Fourier transform (continuous series). Using f̂(ξ, 1), ξ ∈ Ξ_{ℝ}, we will define the continuous part of the spherical Fourier transform. Let us express ξ in parameters (λ, u, v).Then the boundary values (ξ, 1 + i0) will correspond to the λ − sign(λ)i0. The set Ξ_{ℝ} is invariant relative to the action of ℝ^{×} on λ and this action commutates with the action of G_{ℝ} × G_{ℝ} on Ξ_{0}. Let us decompose f̂(ξ) in the Mellin integral on λ:

(40)  f̂_{0}(λ|u, v) = ∫_{0}^{∞} f̃_{e}(ρ|u, v)|λ|^{iρ−1}dρ + ∫_{0}^{∞} f̃_{o}(ρ|u, v)|λ|^{iρ−1} sign(λ)dρ.

We will call f̃_{e}, f̃_{o} by the continuous parts of the spherical Fourier transform(its even and odd parts). It is clear that they transform relative to the action of the group G_{ℝ} × G_{ℝ} by the corresponding representations of the continuous series of representations (cf.(36)).To reproduce the inversion formula (30) for f_{0} using the spherical Fourier transform we need to remark that the operator L corresponds to the multiplication of f̃ on −ρ^{2}.

The computations are similar to the discrete part and we will not reproduce them in all details.We give formulas for the reconstruction in the point e ∈ G_{ℝ}. In this case(in the group parameters) the cycle Γ_{0}(e) consists of matrices

( \begin{matrix}1 + ξ_{1} & ξ_{2} \\ ξ_{3} & 1 − ξ_{1} \end{matrix} ) , ξ_{1}^{2} + ξ_{2}ξ_{3} = 1.

It is a hyperboloid of one sheet. Correspondingly, we have

u = \frac{ξ_{2}}{1 − ξ_{1}} , v = \frac{ξ_{3}}{1 − ξ_{1}} , λ = 1 − ξ_{1}.

The pair u, v ∈ ℝ is the usual parameterization on the hyperboloid Γ_{0}(e).Eventualy we transform the corresponding part of (30) in

(41)  \begin{aligned}f_{0}(e) & = \frac{2}{(πi)^{3}} \{ ∫_{0}^{∞} ∫_{−∞}^{∞} ∫_{0}^{∞} ∫_{−∞}^{∞} ρ^{2}f̃_{e}(ρ|u, v) |\frac{uv + 1}{2}|^{iρ−1} \\ & × \frac{(u − v)(uv − 1)}{(uv + 1)^{3}}dρ ∧ du ∧ dv \\ & + ∫_{0}^{∞} ∫_{−∞}^{∞} ∫_{0}^{∞} ∫_{−∞}^{∞} ρ^{2}f̃_{o}(ρ|u, v) |\frac{uv + 1}{2}|^{iρ−1} sign (\frac{uv + 1}{2}) \\ & × \frac{(u − v)(uv − 1)}{(uv + 1)^{3}}dρ ∧ du ∧ dv\}. \end{aligned}

Remarks.

(1) If instead of horospheres we consider horocycles we will have essentially different formulas not only for the horospherical transform but also for the corresponding spherical Fourier transform (in the continuous part we will have the inversion formula which includes ρ tanh ρ instead ρ^{2}).We will give these formulas in the next publication.

(2) In the next paper we will give several applications: Plancherel formula, models of series, descriptions of projections on series f ↦ f_{±}, f_{0} on the language of complex analysis.

References

[GGi] I.Gelfand and S.Gindikin, Complex manifolds whose skeletons are real semisimple Lie groups and holomorphic discrete series, Functional Anal.Appl. 11 (1977), 19-27.

[GGiGr] I.Gelfand, S.Gindikin and M.Graev, Integral geometry in affine and projective spaces, J.Sov.Math. 18 (1980), 39-67.

[GGr] I.Gelfand and M.Graev, Geometry of homogeneous spaces, representations of groups in homogeneous spaces and related questions of integral geometry, Transl.,II,Amer.Math. Soc. 37 (1964), 351-429.

[GGrV] I.Gelfand, M.Graev and N.Vilenkin, Integral geometry and representations theory., Generalized functions, vol. 5, Academic Press, 1966.

[GGrSh] I.Gelfand, M.Graev and Z.Shapiro, Integral geometry on k-dimensional planes, Functional Anal.Appl. 1 (1967), 14-27.

[Gi1] S.Gindikin, Real integral geometry and complex analysis, Integral geometry, Radon transform and complex analysis . Lecture Notes Math., vol. 1684, Springer, 1996, pp. 70-98.

[Gi2] S.Gindikin, Tube domains in Stein symmetric spaces, Positivity in Lie theory:open problems, de Gruyter, 1998, pp. 81-97.

[Gi3] S.Gindikin, Integral geometry on real quadrics, Amer.Math.Soc.Transl.(2) 169 (1995), 23-31.

Departm. of Math., Hill Center, Rutgers University, 110 Frelinghysen Road, Piscataway, NJ 08854-8019

   E-mail address: gindikin@math.rutgers.edu