On malnormal peripheral subgroups of the fundamental group of a 3-manifold

Let K be a non-trivial knot in the 3-sphere, EK its exterior, GK = π1(EK) its group, and PK = π1(∂EK) ⊂ GK its peripheral subgroup. We show that PK is malnormal in GK , namely that gPKg ∩ PK = {e} for any g ∈ GK with g / ∈ PK , unless K is in one of the following three classes: torus knots, cable knots, and composite knots; these are exactly the classes for which there exist annuli in EK attached to TK which are not boundary parallel (Theorem 1 and Corollary 2). More generally, we characterise malnormal peripheral subgroups in the fundamental group of a compact orientable irreducible 3-manifold of which the boundary is a non-empty union of tori (Theorem 3). Proofs are written with non-expert readers in mind. Half of our paper (Appendices A to D) is a reminder of some three-manifold topology as it flourished before the Thurston revolution. In a companion paper [15], we collect general facts on malnormal subgroups and Frobenius groups, and we review a number of examples.