The Dimits shift is the shift between the threshold of the drift-wave primary instability and the actual onset of turbulent transport in magnetized plasma. It is generally attributed to the suppression of turbulence by zonal flows, but developing a more detailed understanding calls for consideration of specific reduced models. The modified Terry--Horton system has been proposed by St-Onge [J. Plasma Phys. {\bf 83}, 905830504 (2017)] as a minimal model capturing the Dimits shift. Here, we use this model to develop an analytic theory of the Dimits shift and a related theory of the tertiary instability of zonal flows. We show that tertiary modes are localized near extrema of the zonal velocity $U(x)$, where $x$ is the radial coordinate. By approximating $U(x)$ with a parabola, we derive the tertiary-instability growth rate using two different methods and show that the tertiary instability is essentially the primary drift-wave instability modified by the local $U''$. Then, depending on $U''$, the tertiary instability can be suppressed or unleashed. The former corresponds to the case when zonal flows are strong enough to suppress turbulence (Dimits regime), while the latter corresponds to the case when zonal flows are unstable and turbulence develops. This understanding is different from the traditional paradigm that turbulence is controlled by the flow shear $U'$. Our analytic predictions are in agreement with direct numerical simulations of the modified Terry--Horton system.