Dynamical instability is studied in a deterministic dynamical system of Hamiltonian type composed of a tracer particle in a fluid of many particles. The tracer and fluid particles are hard balls (disks, in two dimensions, or spheres, in three dimensions) undergoing elastic collisions. The dynamical instability is characterized by the spectrum of Lyapunov exponents. The tracer particle is shown to dominate the Lyapunov spectrum in the neighborhoods of two limiting cases: the Lorentz-gas limit in which the tracer particle is much lighter than the fluid particles and the Rayleigh-flight limit in which the fluid particles have a vanishing radius and form an ideal gas. In both limits, a gap appears in the Lyapunov spectrum between the few largest Lyapunov exponents associated with the tracer and the rest of the Lyapunov spectrum.