ON A QUESTION OF A. KOLDOBSKY

ALEXEY GONCHAROV AND DMITRY RYABOGIN

Abstract. We construct an example of a non-convex star-shaped origin-symmetric body D ⊂ R^{3} such that its section function A_{D, ξ}(t) := area(D ∩ {ξ^{⊥} + tξ}) is decreasing in t ≥ 0 for every fixed direction ξ ∈ S^{2}.

1. Introduction

Let K ⊂ R^{n} be a convex body, and let ξ^{⊥} := {x ∈ R^{n} : x·ξ = 0}, where ξ ∈ S^{n−1}. The famous Theorem of Brunn states that the section function t ∈ R → A_{K, ξ}^{1/(n−1)}(t), A_{K, ξ}(t) := vol_{n−1}(K ∩ {ξ^{⊥} + tξ}), is concave on its support [1], [2]. One consequence of this is that if K is centrally symmetric, the largest slice perpendicular to a given direction is the central slice. Another consequence is that if K is origin symmetric, the function A_{K, ξ}(t) is decreasing in t ≥ 0 ∀ξ ∈ S^{n−1}.

It is completely natural to study converse statements. For example, in [3], E. Makai, H. Martini and T. Odor proved the following result. Let K be a convex body and assume that for every direction ξ the maximal section (among all perpendicular to ξ) contains the origin. Then the body is symmetric with respect to the origin.

The present paper deals with another problem motivated by Brunn’s theorem. The question was posed by A. Koldobsky and reads as follows. Let D ⊂ R^{n} be an origin symmetric star body. Assume that A_{D, ξ}(t) := vol_{n−1}(D ∩ {ξ^{⊥} + tξ}) is decreasing in t ≥ 0 ∀ξ ∈ S^{n−1}. Is it true that D is convex?

In this note we give a negative answer to the question.

Theorem 1. Let l, r, R be positive real numbers, satisfying

(1)  r ≤ R/2,  l ≤ r/2,

and let

(2)  h = \sqrt{r^{2} − l^{2}},  H = \sqrt{R^{2} − l^{2}}.

We define

B = {(x, y, z) ∈ R^{3} : x^{2} + y^{2} + z^{2} ≤ r^{2}},
B_{+} = {(x, y, z) ∈ R^{3} : x^{2} + y^{2} + (z − H − h)^{2} ≤ R^{2}},
B_{−} = {(x, y, z) ∈ R^{3} : x^{2} + y^{2} + (z + H + h)^{2} ≤ R^{2}}.

Date: September 8, 2008.

1991 Mathematics Subject Classification. Primary 42B20. Secondary 42E30.

Key words and phrases. Brunn-Minkowski theorem, sections of convex bodies.

Goncharov’s research was partially supported by the CBMS/NSF grant DMS-0532656. Ryabogin’s research was partially supported by the FRG/NSF grant DMS-0652684.

Then the body

D = B \ ( B_{+} ∪ B_{−})

is a non-convex star-shaped origin-symmetric body of revolution, such that its section function A_{D, ξ}(t) := area(D ∩ {ξ^{⊥} + tξ}) is decreasing in t ≥ 0 for every fixed direction ξ ∈ S^{2}.

Since D is a body of revolution, it is enough to consider the section function A_{D, ξ}(t) for ξ = (− cos α, 0, sin α), 0 ≤ α ≤ π/2.

Our proof is elementary. For each element of the partition

[0, π/2] = {0} ∪ (0, α_{0}) ∪ {α_{0}} ∪ (α_{0}, π/2 − α_{0}) ∪ {π/2 − α_{0}} ∪
∪(π/2 − α_{0}, π/2 − μ) ∪ {π/2 − μ} ∪ (π/2 − μ, π/2) ∪ {π/2},
α_{0} = arcsin (l/r), μ = arcsin (l/R),

we write out the formula for the section function S_{α}(t) := A_{D, (− cos α, 0, sin α)}(t). Then we check the decreasing (in t) behavior of S_{α}(t) by showing that S′_{α}(t) < 0 for 0 ≤ t ≤ r except finitely many points.

2. Auxiliary results

Observe that (1) and (2) imply

μ ≤ arcsin (1/4), sin α_{0} = l/r ≥ 2l/R = 2 sin μ,

and

(3)  α_{0} ≤ \frac{π}{6} , μ < \frac{π}{12}, α_{0} > μ;
(4)  l = r sin α_{0}, h = r cos α_{0}, h ≥ l\sqrt{3};
(5)  l = R sin μ, H = R cos μ, H ≥ l\sqrt{15}.

The proof of the following statements is straightforward, and we leave it to the reader.

Lemma 1. Let u(t), v(t) be two continuously-differentiable functions on (a, b), and let

u(t) ≥ 0, v(t) > 0, u(t) < v(t), a < t < b.

Then the function

f(t) ≡ v(t) arcsin \sqrt{\frac{u(t)}{v(t)}} − \sqrt{u(t)}\sqrt{v(t) − u(t)}, a < t < b,

is continuously-differentiable, and

f′(t) = v′(t) arcsin \sqrt{\frac{u(t)}{v(t)}} − 2\sqrt{u(t)}(\sqrt{v(t) − u(t)})′, a < t < b.

Definition. Let C\sqrt{v} = \{x ∈ R^{2} : x_{1}^{2} + x_{2}^{2} ≤ v\} be a disc of radius \sqrt{v}, and let y ∈ R^{2} : y_{1}^{2} + y_{2}^{2} = 1. A subset (2\sqrt{u}, \sqrt{v}), u < v, of C\sqrt{v} is called a minor (major) segment, bounded by a chord of length 2\sqrt{u} and an arc of a circumference of radius \sqrt{v}, if

(2\sqrt{u}, \sqrt{v}) = \{x ∈ C\sqrt{v} : x · y ≥ (≤)\sqrt{v − u}\}.

Lemma 2. Let S be an area of a segment (2\sqrt{u}, \sqrt{v}). Then

S = v arcsin(\sqrt{u/v}) − \sqrt{u}\sqrt{v − u},

provided the segment is minor, and

S = πv − v arcsin(\sqrt{u/v}) + \sqrt{u}\sqrt{v − u},

provided the segment is major.

3. α = 0.

It is enough to consider the section D ∩ {(−1, 0, 0)^{⊥} − t(−1, 0, 0)}, 0 ≤ t ≤ r. We consider two cases.

Case 1. The section is a disc of radius \sqrt{r^{2} − t^{2}}, t < l, without four minor segments. Two of the segments are (2\sqrt{l^{2} − t^{2}}, \sqrt{r^{2} − t^{2}}). The other two are (2\sqrt{l^{2} − t^{2}}, \sqrt{R^{2} − t^{2}}).

Case 2. The section is a disc of radius \sqrt{r^{2} − t^{2}}, l ≤ t ≤ r.

The second case is obvious and we consider the first one. We apply Lemmata 1 and 2 to functions

u(t) ≡ l^{2} − t^{2},  v(t) ≡ r^{2} − t^{2},  0 < t < l,

and then to functions u(t) and

v(t) ≡ R^{2} − t^{2}, 0 < t < l.

By Lemma 2, we have

S_{0}(t) = π(r^{2} − t^{2}) − 2(r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − t^{2}}{r^{2} − t^{2}}} + 2\sqrt{l^{2} − t^{2}}\sqrt{r^{2} − l^{2}}−

2(R^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − t^{2}}{R^{2} − t^{2}}} + 2\sqrt{l^{2} − t^{2}}\sqrt{R^{2} − l^{2}},

provided 0 ≤ t < l. By Lemma 1, we obtain:

S′_{0}(t) = 2t (−π + 2 arcsin \sqrt{\frac{l^{2} − t^{2}}{r^{2} − t^{2}}} + 2 arcsin \sqrt{\frac{l^{2} − t^{2}}{R^{2} − t^{2}}} ) , 0 < t < l.

Now, the functions

t → \frac{l^{2} − t^{2}}{r^{2} − t^{2}} , t → \frac{l^{2} − t^{2}}{R^{2} − t^{2}}

are decreasing on 0 ≤ t ≤ l, and (3) implies

S′_{0}(t) ≤ 2t(−π + 2 arcsin(l/r) + 2 arcsin(l/R)) = 2t(−π + 2α_{0} + 2μ) < −πt,

0 < t < l. Thus, S_{0}(t) is decreasing on 0 ≤ t ≤ l, and, hence, S_{0}(t) is decreasing on 0 ≤ t ≤ r.

4. 0 < α < α_{0}.

We have

(6)  tan α < l/h.

We will use the following

Lemma 3. We have

(7)  h/ cos α − t tan α > 0,  0 ≤ t ≤ l cos α + h sin α,
(8)  H cos α − (h sin α + t) tan α > 0,  0 ≤ t ≤ l cos α − h sin α,
(9)  H cos α − (h sin α − t) tan α > 0,  0 ≤ t ≤ l cos α + h sin α.

Proof. Relation (7) follows from two observations: the function

t → h/ cos α − t tan α

is decreasing and is zero for t = h/ sin α; and secondly we have

l cos α + h sin α < h/ sin α,

(use (6) and the last relation in (4)). Relation (8) is a consequence of α < α_{0}, (3), the fact that the function

t → H cos α − (h sin α + t) tan α

is decreasing and for t = l cos α − h sin α is equal to

H cos α − l sin α = R cos(α + μ).

Since the function

t → H cos α − (h sin α − t) tan α,

is increasing, inequalities

H cos α − (h sin α − t) tan α > H cos α − (h sin α + t) tan α, t > 0,

and (8) yield the last relation of the lemma. □

We write out the section function for three separate cases.

Case 1: t < l cos α − h sin α. Our section is a disc of radius \sqrt{r^{2} − t^{2}} without four minor segments:

(2\sqrt{l^{2} − (h tan α + t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}),

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}),

(2\sqrt{l^{2} − (h tan α + t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α + t)^{2}}),

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}).

Applying Lemma 2 (with v(t) = r^{2} − t^{2}, u(t) = l^{2} − (h tan α + t/ cos α)^{2}; v(t) = r^{2} −t^{2}, u(t) = l^{2} −(h tan α−t/ cos α)^{2}; v(t) = R^{2} −(H sin α+h sin α+t)^{2}, u(t) = l^{2} − (h tan α+t/ cos α)^{2}; v(t) = R^{2}−(H sin α+h sin α−t)^{2}, u(t) = l^{2}−(h tan α−t/ cos α)^{2} correspondingly), and the previous lemma, we have:

S_{α}(t) = π(r^{2} − t^{2}) − (r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α + t/ cos α)^{2}}{r^{2} − t^{2}}} +

+\sqrt{l^{2} − (h tan α + t/ cos α)^{2}}(h/ cos α + t tan α)−

−(r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(h/ cos α − t tan α)−

−(R^{2} − (H sin α + h sin α + t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α + t/ cos α)^{2}}{R^{2} − (H sin α + h sin α + t)^{2}}} +

+\sqrt{l^{2} − (h tan α + t/ cos α)^{2}}(H cos α − (h sin α + t) tan α)−

−(R^{2} − (H sin α + h sin α − t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(H cos α − (h sin α − t) tan α).

Case 2: l cos α − h sin α ≤ t < l cos α + h sin α = r sin(α + α_{0}) < r. Our section is a disc of radius \sqrt{r^{2} − t^{2}} without two minor segments:

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}),

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}). Applying Lemma 2 (with v(t) = r^{2}−t^{2}, u(t) = l^{2}−(h tan α−t/ cos α)^{2}; v(t) = R^{2}−(H sin α+h sin α−t)^{2}, u(t) = l^{2} − (h tan α − t/ cos α)^{2}), and the previous lemma, we have:

S_{α}(t) = π(r^{2} − t^{2}) − (r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(h/ cos α − t tan α)−

−(R^{2} − (H sin α + h sin α − t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(H cos α − (h sin α − t) tan α).

Case 3: t ≥ l cos α + h sin α. Our section is a disc of radius \sqrt{r^{2} − t^{2}}.

S_{α}(t) = π(r^{2} − t^{2}).

Now we compute the derivative S′_{α}(t) and show that S′_{α}(t) < 0 ∀t ∈ [0, r]. It is enough to consider only first two cases.

Observe that

\frac{d}{dt}(h/ cos α ± t tan α) + \frac{d}{dt}(H/ cos α − (h sin α ± t) tan α) = 0.

6 ALEXEY GONCHAROV AND DMITRY RYABOGIN

Then, applying Lemma 1 (use (9) – (15)), we have

S′_{α}(t) = −2πt + 2t arcsin \sqrt{\frac{l^{2} − (h tan α + t/ cos α)^{2}}{r^{2} − t^{2}}} +

+2t arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+2(H sin α + h sin α + t) arcsin \sqrt{\frac{l^{2} − (h tan α + t/ cos α)^{2}}{R^{2} − (H sin α + h sin α + t)^{2}}} −

−2(H sin α + h sin α − t) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} ,

provided 0 ≤ t < l cos α − h sin α, and

S′_{α}(t) = −2πt + 2t arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} −

−2(H sin α + h sin α − t) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} ,

provided l cos α − h sin α ≤ t < l cos α + h sin α.

To show S′_{α}(t) < 0 ∀t ∈ [0, r] we use Lemma 4. For 0 ≤ t ≤ l cos α − h sin α, we have

(10)  arcsin \sqrt{\frac{l^{2} − (h tan α + t/ cos α)^{2}}{r^{2} − t^{2}}} < π/6,
(11)  arcsin \sqrt{\frac{l^{2} − (h tan α ± t/ cos α)^{2}}{R^{2} − (H sin α + h sin α ± t)^{2}}} < π/6,
(12)  \frac{l^{2} − (h tan α + t/ cos α)^{2}}{R^{2} − (H sin α + h sin α + t)^{2}} ≤ \frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}} .

For 0 ≤ t ≤ l cos α + h sin α, we have

(13)  arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} < π/2,
(14)  arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} < π/2.

We postpone for a moment the proof of the lemma and show how it implies the claim. For 0 ≤ t < l cos α − h sin α, relations (10) - (13) yield

S′_{α}(t) = −2t(π − arcsin \sqrt{\frac{l^{2} − (h tan α + t/ cos α)^{2}}{r^{2} − t^{2}}} −

− arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}}

− arcsin \sqrt{\frac{l^{2} − (h tan α + t/ cos α)^{2}}{R^{2} − (H sin α + h sin α + t)^{2}}} − arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} )

+ 2(H sin α + h sin α)( arcsin \sqrt{\frac{l^{2} − (h tan α + t/ cos α)^{2}}{R^{2} − (H sin α + h sin α + t)^{2}}} −

arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} ) < 0.

Now, for l cos α − h sin α < t < l cos α + h sin α, relations (13), (14) give

S′_{α}(t) = 2t( − π + arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+ arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} )

− 2(H sin α + h sin α) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} < 0.

We remind that

S′_{α}(t) = −2πt,

provided l cos α + h sin α < t ≤ r. Since S_{α}(t) is continuous on 0 ≤ t ≤ r, it is decreasing on 0 ≤ t ≤ r for any fixed α ∈ (0, α_{0}). It remains to prove the lemma.

Proof. We will use the following relations, that could be checked directly.

(15) r^{2} − t^{2} − l^{2} + (h tan α ± t/ cos α)^{2} = (h/ cos α ± t tan α)^{2},  −∞ < t < +∞,

(16) R^{2} − (H sin α + h sin α ± t)^{2} − l^{2} + (h tan α ± t/ cos α)^{2} =

= (H cos α − (h sin α ± t) tan α)^{2}, −∞ < t < +∞.

We prove (10). Since the function

t → \frac{l^{2} − (h tan α + t/ cos α)^{2}}{r^{2} − t^{2}}

is decreasing on 0 ≤ t < r (its derivative

\frac{−2ht^{2} tan α/ cos α − 2(r^{2}/ cos^{2} α − l^{2} + h^{2} tan^{2} α)t − 2r^{2}h tan α/ cos α}{(r^{2} − t^{2})^{2}}

is negative), we have

arcsin \sqrt{\frac{l^{2} − (h tan α + t/ cos α)^{2}}{r^{2} − t^{2}}} ≤ arcsin \sqrt{\frac{l^{2} − h^{2} tan^{2} α}{r^{2}}} < arcsin(l/r) = α_{0}.

It remains to use (3) to obtain (10).

Relations (15), (7) imply

\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}} < 1, 0 ≤ t ≤ l cos α + h sin α.

This gives (13). Similarly, (16), (9) yield (14).

To show (11) we observe that for t > 0,

(H sin α + h sin α + t)^{2} > (H sin α + h sin α − t)^{2},

gives

\frac{1}{R^{2} − (H sin α + h sin α − t)^{2}} < \frac{1}{R^{2} − (H sin α + h sin α + t)^{2}} .

Since the function

t → \frac{1}{R^{2} − (H sin α + h sin α + t)^{2}}

is increasing, relation (5) implies:

\frac{l^{2} − (h tan α ± t/ cos α)^{2}}{R^{2} − (H sin α + h sin α ± t)^{2}} < \frac{l^{2}}{R^{2} − (H sin α + h sin α + t)^{2}} ≤

≤ \frac{l^{2}}{R^{2} − (H sin α + l cos α)^{2}} = \frac{l^{2}}{R^{2} − R^{2} sin^{2}(μ + α)}, 0 ≤ t ≤ l cos α − h sin α.

Moreover, (3) and (1) yield

\frac{l^{2} − (h tan α ± t/ cos α)^{2}}{R^{2} − (H sin α + h sin α ± t)^{2}} < \frac{l^{2}}{R^{2} cos^{2}(π/4)} = \frac{2l^{2}}{R^{2}} < 1/8.

This gives (11). Finally, to prove (12) it is enough to check that

4t sin α(l^{2}h + l^{2}H − R^{2}h/ cos^{2} α + H^{2}h tan^{2} α + Hh^{2} tan^{2} α − t^{2}H/ cos^{2} α) ≤ 0,

or

l^{2}h + l^{2}H − R^{2}h/ cos^{2} α + H^{2}h tan^{2} α + Hh^{2} tan^{2} α < 0.

Since

R^{2} = l^{2} + H^{2}, tan α < l/h, l ≤ h/\sqrt{3}, l ≤ H/\sqrt{15},

then

l^{2}h + l^{2}H − R^{2}h/ cos^{2} α + H^{2}h tan^{2} α + Hh^{2} tan^{2} α =
= l^{2}h + l^{2}H − (l^{2} + H^{2})(1 + tan^{2} α)h + H^{2}h tan^{2} α + Hh^{2} tan^{2} α =
= −H^{2}h + l^{2}H − l^{2}h tan^{2} α + Hh^{2} tan^{2} α < 2l^{2}H − H^{2}h <

< 2\frac{h}{\sqrt{3}} \frac{H}{\sqrt{15}}H − H^{2}h = (\frac{2}{3\sqrt{5}} − 1) H^{2}h < 0.

The lemma is proved. □

5. α = α_{0}.

We have

sin α = l/r, cos α = h/r.

It is clear that our section is a disc of radius r, provided t = 0.

Case 1: 0 < t < l cos α + h sin α = 2lh/r < r. It is a disc of radius \sqrt{r^{2} − t^{2}} without two segments

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}),

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}).

Case 2: 2lh/r ≤ t ≤ r. It is a disc of radius \sqrt{r^{2} − t^{2}}.

Since relations (7), (9), (11), (12) remain valid for α = α_{0}, we have

S_{α}(0) = πr^{2};

S_{α}(t) = π(r^{2} − t^{2}) − (r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(h/ cos α − t tan α) −

−(R^{2} − (H sin α + h sin α − t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} +

+ \sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(H cos α − (h sin α − t) tan α),

provided 0 < t < 2lh/r,

S_{α}(t) = π(r^{2} − t^{2}),

provided 2lh/r ≤ t ≤ r;

S′_{α}(t) = 2t( − π + arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+ arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} )−

−2(H sin α + h sin α) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} < 0,

provided 0 < t < 2lh/r. Thus, our function S_{α}(t) is decreasing on 0 ≤ t ≤ 2lh/r, hence, on 0 ≤ t ≤ r.

6. α_{0} < α < π/2 − α_{0}.

Case 1: 0 ≤ t ≤ h sin α − l cos α. Our section is a disc of radius \sqrt{r^{2} − t^{2}}.

Case 2: h sin α − l cos α < t < h sin α + l cos α = r sin(α + α_{0}) < r. It is a disc of radius \sqrt{r^{2} − t^{2}} without two minor segments

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}),

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}).

Case 3: h sin α + l cos α ≤ t ≤ r. It is a disc of radius \sqrt{r^{2} − t^{2}}.

Lemma 5. Estimates (7) and (9) remain true, provided

h sin α − l cos α < t < h sin α + l cos α, α_{0} < α < π/2 − α_{0}.

Proof. We prove (7). The function

t → h/ cos α − t tan α

is decreasing and is zero for t = h/ sin α. Moreover, we have

l cos α + h sin α < h/ sin α,

otherwise

h/ sin α < l cos α + h sin α

leads (use (4))to a contradiction

\{ \begin{matrix}h cos α − l sin α = r cos(α + α_{0}) < 0, \\ 0 < α + α_{0} < π/2. \end{matrix}

To check (9) we observe that its left-hand side is increasing in t, and (use (5), (3)) for t = h sin α − l cos α takes the value

H cos α − l sin α = R cos(μ + α) > R sin(α_{0} − μ) > 0.
□

Thus, we may apply Lemma 2 and Lemma 1. It is clear that

S_{α}(t) = π(r^{2} − t^{2}),

provided 0 ≤ t ≤ h sin α − l cos α. If h sin α − l cos α < t < h sin α + l cos α, we have

S_{α}(t) = π(r^{2} − t^{2}) − (r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(h/ cos α − t tan α)−

−(R^{2} − (H sin α + h sin α − t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(H cos α − (h sin α − t) tan α).

We also have

S_{α}(t) = π(r^{2} − t^{2}),

provided h sin α − l cos α ≤ t ≤ r. Now, we use (13), (14) to obtain

S′_{α}(t) = 2t( − π + arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+ arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} )−

−2(H sin α + h sin α) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} < 0,

provided h sin α − l cos α < t < h sin α + l cos α.

Thus, S_{α}(t) is decreasing on [h sin α − l cos α, h sin α + l cos α], and, therefore, on 0 ≤ t ≤ r.

7. α = π/2 − α_{0}.

We have

sin α = h/r, cos α = l/r.

Our section is a disc of radius \sqrt{r^{2} − t^{2}}, provided 0 ≤ t ≤ h sin α − l cos α = \frac{h^{2} − l^{2}}{r}.

If h sin α − l cos α < t < h sin α + l cos α = \frac{h^{2} + l^{2}}{r} = r, it is a disc of radius \sqrt{r^{2} − t^{2}}, without two minor segments,

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}),

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}). S_{α}(t) is defined as in the previous section (observe that

lim_{t→r− 0} \frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}} = lim_{t→r− 0} \frac{l^{4} − (h^{2} − rt)^{2}}{l^{2}(r^{2} − t^{2})} = 1 ).

8. π/2 − α_{0} < α < π/2 − μ.

We observe that

h sin α − l cos α ≤ h/ sin α ≤ h sin α + l cos α.

The first inequality is obvious, the second one follows from

h cos α − l sin α = r cos(α + α_{0}) < 0.

We consider three cases.

Case1: 0 ≤ t ≤ h sin α − l cos α. Our section is a disc of radius \sqrt{r^{2} − t^{2}}.

Case2: h sin α − l cos α < t < h/ sin α. It is a disc of radius \sqrt{r^{2} − t^{2}} without two minor segments

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}),

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}).

Case3: h/ sin α ≤ t < h sin α + l cos α, our section is a disc of radius \sqrt{r^{2} − t^{2}} without major and minor segments

(2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}), (2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}). It is a point, provided t = h sin α + l cos α. We will use the following

Lemma 6. We have

(17) h/ cos α − t tan α > 0, h sin α − l cos α < t < h/ sin α, h/ cos α − t tan α < 0, h/ sin α < t < h sin α + l cos α;

(18)  arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} < π/2,

provided h sin α − l cos α < t < h/ sin α;

(19)  \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} > \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} ,

provided h/ sin α < t < h sin α + l cos α. Finally, relation (9) is true for h sin α − l cos α < t < h sin α + l cos α.

Proof. Relation (17) is clear, (18) follows from (15), (17). To check (19), it is enough to prove that

r^{2} − t^{2} < R^{2} − (H sin α + h sin α − t)^{2},

or (use R^{2} − r^{2} = H^{2} − h^{2})

(H sin α + h sin α)^{2} < H^{2} − h^{2} + 2t(H sin α + h sin α).

The right-hand side of the last inequality is increasing in t, and taking t = h/ sin α, we have

H^{2} − h^{2} + 2h(H + h) = (H + h)^{2} > (H sin α + h sin α)^{2}.

This gives (19). To prove (9) it is enough to observe that

H cos α − l sin α = R cos(α + μ) > 0.
□

Now, taking into account the previous Lemma, and applying Lemma 2 and Lemma 1, we have

S_{α}(t) = π(r^{2} − t^{2}),

provided 0 ≤ t ≤ h sin α − l cos α,

S_{α}(t) = π(r^{2} − t^{2}) − (r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(h/ cos α − t tan α)−

−(R^{2} − (H sin α + h sin α − t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(H cos α − (h sin α − t) tan α),

provided h sin α − l cos α < t < h/ sin α, and

S_{α}(t) = (r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} −

−\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(t tan α − h/ cos α)−

−(R^{2} − (H sin α + h sin α − t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} +

+\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(H cos α − (h sin α − t) tan α),

provided h/ sin α ≤ t ≤ h sin α + l cos α.

Taking the derivative, and using (18), (19), we obtain

S′_{α}(t) = 2t( − π + arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+ arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} )−

−2(H sin α + h sin α) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} < 0,

provided h sin α − l cos α < t < h/ sin α. If h/ sin α < t < h sin α + l cos α,

S′_{α}(t) = −2t arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} −

−2(H sin α + h sin α − t) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} .

To show that the last expression is negative, we use (19):

S′_{α}(t) = 2t( arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} −

-

arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} )−

−2(H sin α + h sin α) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} < 0,

provided h/ sin α < t < h sin α+l cos α. Thus, our section function S_{α}(t) is decreasing for h/ sin α < t < h sin α + l cos α, and, hence, for 0 ≤ t ≤ h sin α + l cos α.

9. α = π/2 − μ.

The section function S_{α}(t) is defined as in the previous section, (observe that

lim_{t→h sin α−l cos α+0}(R^{2} − (H sin α + h sin α − t)^{2}) = 0,

lim_{t→h sin α−l cos α+0} \frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}} = 1 ).

10. π/2 − μ < α < π/2.

Lemma 7. The following chain of inequalities is true:

0 < H sin α + h sin α − R < h sin α − l cos α < h sin α + H sin α − \frac{H}{sin α} <

\frac{h}{sin α} < h sin α + l cos α.

Proof. The first inequality follows from sin α > cos μ, (5), (4), and (3):

(20) H sin α + h sin α − R > h sin α + H cos μ − R =

= h sin α + R cos^{2} μ − R = h sin α − l sin μ > 0,

The second one is a consequence of R sin(α + μ) < R and (5). The third inequality follows from H cos α < l sin α, or H cos α − l sin α = R cos(α + μ) < 0. The fourth inequality is obvious. The last one follows from

h cos α − l sin α = r cos(α + α_{0}) < 0.
□

We split [0, h sin α + l cos α] into 5 intervals, defined by inequalities of the previous lemma. Our section is

Case 1: a disc of radius \sqrt{r^{2} − t^{2}}, provided 0 ≤ t ≤ H sin α + h sin α − R. Case 2: a disc of radius \sqrt{r^{2} − t^{2}}, without a circle of radius \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}, provided H sin α + h sin α − R < t ≤ h sin α − l cos α. Case 3: a disc of radius \sqrt{r^{2} − t^{2}} without a minor and major segments (2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}), (2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}), provided h sin α − l cos α < t ≤ h sin α + H sin α − H/ sin α. Case 4: a disc of radius \sqrt{r^{2} − t^{2}}, without two minor segments (2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}) (2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}), provided h sin α + H sin α − H/ sin α < t < h/ sin α. Case 5: a disc of radius \sqrt{r^{2} − t^{2}} without major and minor segments: (2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{r^{2} − t^{2}}), (2\sqrt{l^{2} − (h tan α − t/ cos α)^{2}}, \sqrt{R^{2} − (H sin α + h sin α − t)^{2}}), provided h/ sin α ≤ t < h sin α + l cos α. If t = h sin α + l cos α, the section is a point.

Before we start considering the section function in the above cases, we prove one more lemma.

Lemma 8. We have

(21)  H cos α − (h sin α − t) tan α < 0,

provided

h sin α − l cos α < t < h sin α + H sin α − H/ sin α,
(22)  H cos α − (h sin α − t) tan α > 0,

provided

h sin α + H sin α − H/ sin α < t < h sin α + l cos α,

(23)  arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} < π/2,

provided

h sin α + H sin α − H/ sin α < t < h sin α + l cos α,

and

(24) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} < arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} ,

provided

h sin α − l cos α < t < h sin α + H sin α − H/ sin α.

Proof. To prove (21) and (22) we observe that the function

t → f(t) = H cos α − (h sin α − t) tan α

is increasing, and

f(h sin α − l cos α) = H cos α − l sin α = R cos(μ + α) < 0,
f(h sin α + H sin α − H/ sin α) = 0,
f(h sin α + l cos α) = H cos α + l sin α > 0,

Relation (23) is a consequence of (16). To prove (24), it is enough to show

R^{2} − (H sin α + h sin α − t)^{2} < r^{2} − t^{2},

or (R^{2} − r^{2} = H^{2} − h^{2})

H^{2} − h^{2} − (H sin α + h sin α)^{2} + 2(H sin α + h sin α)t < 0.

The left-hand side of the last inequality is increasing (in t), and for t = h sin α + H sin α − H/ sin α we have

H^{2} − h^{2} + (H sin α + h sin α)^{2} − 2H(H + h) = (H sin α + h sin α)^{2} − (H + h)^{2} < 0.

□

16 ALEXEY GONCHAROV AND DMITRY RYABOGIN

Taking into account (17), (21), (18), (23), (19), we have

Case 1 : S_{α}(t) = π(r^{2} − t^{2}),

provided 0 ≤ t ≤ H sin α + h sin α − R,

Case 2 : S_{α}(t) = π(r^{2} − t^{2}) − π(R^{2} − (H sin α + h sin α − t)^{2}) =

= π(r^{2} − R^{2} + (H sin α + h sin α)(H sin α + h sin α − 2t)),

provided H sin α + h sin α − R < t ≤ h sin α − l cos α,

Case 3 : S_{α}(t) = π(r^{2} − R^{2} + (H sin α + h sin α)(H sin α + h sin α − 2t))−

− (r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+ \sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(h/ cos α − t tan α)+

+ (R^{2} − (H sin α + h sin α − t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} −

− \sqrt{l^{2} − (h tan α − t/ cos α)^{2}}((h sin α − t) tan α − H cos α),

provided h sin α − l cos α < t ≤ h sin α + H sin α − H/ sin α,

Case 4 : S_{α}(t) = π(r^{2} − t^{2}) − (r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+ \sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(h/ cos α − t tan α)−

− (R^{2} − (H sin α + h sin α − t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} +

+ \sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(H cos α − (h sin α − t) tan α),

provided h sin α + H sin α − H/ sin α < t < h/ sin α, and

Case 5 : S_{α}(t) = (r^{2} − t^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} −

− \sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(t tan α − h/ cos α)−

− (R^{2} − (H sin α + h sin α − t)^{2}) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} +

+ \sqrt{l^{2} − (h tan α − t/ cos α)^{2}}(H cos α − (h sin α − t) tan α),

provided h/ sin α ≤ t ≤ h sin α + l cos α.

We consider Cases 3, 4 and 5, cases 1 and 2 are clear.

Case 3: For h sin α − l cos α < t < h sin α + H sin α − H/ sin α, we have

S′_{α}(t) = −2π(H sin α + h sin α) + 2t arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+ 2(H sin α + h sin α − t) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} =

−2 (H sin α + h sin α)(π − arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} )+

2 t ( arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} −arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} ) < 0,

due to (24).

Case 4: For h sin α + H sin α − H/ sin α < t < h/ sin α, we have

S′_{α}(t) = 2t (−π + arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} +

+ arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} ) −

− 2(H sin α + h sin α) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} < 0.

Case 5: For h/ sin α < t < h sin α + l cos α,

S′_{α}(t) = 2t (arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} −

− arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{r^{2} − t^{2}}} ) −

− 2(H sin α + h sin α) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} <

2(t − (H sin α + h sin α)) arcsin \sqrt{\frac{l^{2} − (h tan α − t/ cos α)^{2}}{R^{2} − (H sin α + h sin α − t)^{2}}} < 0,

where the last inequality follows from t − (H sin α + h sin α) < h sin α + l cos α − (H sin α + h sin α) = l cos α − H sin α =

R sin μ cos α − R sin α cos μ = R sin(μ − α) < 0,

(use (3) and (5)).

11. α = π/2.

Case 1. Our section is a disc of radius \sqrt{r^{2} − t^{2}}, provided 0 ≤ t ≤ h + H − R. Case 2. It is a ring, bounded by two concentric discs of radii \sqrt{r^{2} − t^{2}} and

\sqrt{R^{2} − (H + h − t)^{2}}, provided h + H − R < t < h.

Case 3. It is a disc of radius l, provided t = h.

Thus,

S_{α}(t) = π(r^{2} − t^{2}),

provided 0 ≤ t ≤ H + h − R, and

S_{α}(t) = π(r^{2} − t^{2}) − π(R^{2} − (H + h − t)^{2}) =

= π(r^{2} − R^{2} + (H + h)(H + h − 2t)),

provided H + h − R < t ≤ h.

It is clear that S_{α}(t) is decreasing.

Acknowledgement. We are very grateful to the referee for valuable remarks that helped to improve the paper.

References

[1] K. Ball, An Elementary Introduction to Modern Convex Geometry, Flavors of Geometry, MSRI Publications, Volume 31, 1997.

[2] R. J. Gardner, The Brunn-Minkowski Inequality, Bull. Amer. Math. Soc. 39 (2002), 355-405.

[3] E. Makai, H. Martini, T. Odor, Maximal sections and centrally symmetric bodies, Mathematika 47 (2000), 19-30.

Alexey Goncharov, Department of Mechanics and Mathematics, Rostov (South Federal) State University, Rostov on Don, 344090, Russia

Dmitry Ryabogin, Department of Mathematics, Kent State University, Kent, OH 44242, USA

  E-mail address: ryabogin@math.kent.edu