Abstract:We study the stability of steady nonlinear waves on the surface of an infinitely deep fluid ( Lamb 1964 , Moiseev 1960 ). In section 1, the equations of hydrodynamics for an ideal fluid with a free surface are transformed to canonical variables: the shape of the surface η( r, t) and the hydrodynamic potential $\varPsi ({\boldsymbol{r}},t) $ at the surface are expressed in terms of these variables. By introducing canonical variables, we can consider the problem of the stability of surface waves as part of the more general problem of nonlinear waves in media with dispersion ( Akhmanov 1964 , Zakharov 1965 ). The resuits of the rest of the paper are also easily applicable to the general case. In section 2, using a method similar to van der Pohl's method, we obtain simplified equations describing nonlinear waves in the small amplitude approximation. These equations are particularly simple if we assume that the wave packet is narrow. The equations have an exact solution which approximates a periodic wave of finite amplitude. In section 3 we investigate the instability of periodic waves of finite amplitude. Instabilities of two types are found. The first type of instability is destructive instability, similar to the destructive instability of waves in a plasma ( Oraevskii & Sagdeev 1963 , Oraevskii 1964 ), In this type of instability, a pair of waves is simultaneously excited, the sum of the frequencies of which is a multiple of the frequency of the original wave. The most rapid destructive instability occurs for capillary waves and the slowest for gravitational waves. The second type of instability is the negative-pressure type, which arises because of the dependence of the nonlinear wave velocity on the amplitude; this results in an unbounded increase in the percentage modulation of the wave. This type of instability occurs for nonlinear waves through any media in which the sign of the second derivative in the dispersion law with respect to the wave number (d ² ω/d k ²) is different from the sign of the frequency shift due to the nonlinearity. As announced by A. N. Litvak and V. I. Talanov ( 1967 ), this type of instability was independently observed for nonlinear electromagnetic waves.