This paper presents a new design method based on a robust-control strategy in the form of a linear matrix inequality (LMI) approach for a passive tuned mass damper (TMD), which is one of the common passive-control devices for structural vibration control. To apply the robust control theory, we first present an equivalent expression that describes a passive TMD as an active TMD. Then, some LMI-based condition is derived that not only guarantees robust stability but also allows us to adjust the robust performance. In particular, this paper considers the transfer function from a seismic-wave input to structural responses. Unlike other methods, this method formulates the problem to be a convex optimization problem that ensures a global optimal solution and considers uncertainties of mass, damping, and stiffness of a structure for designing a TMD. Numerical example uses both a single-degree-of-freedom (SDOF) and 10DOF models and seismic waves. The simulation results demonstrated that the TMD that is designed by the presented method has good control performance even if the structural model includes uncertainties, which are the modeling errors.