The fast and faithful preparation of the ground state of quantum systems is a challenging but crucial task for several applications in the realm of quantum-based technologies. Decoherence limits the maximum time-window allowed to an experiment to faithfully achieve such desired states. This is of particular significance in systems featuring a quantum phase transition, where the vanishing energy gap challenges an adiabatic ground state preparation. We show that a bang-bang protocol, consisting of a time evolution under two different values of an externally tunable parameter, allows for a high-fidelity ground state preparation in evolution times no longer than those required by the application of standard optimal control techniques, such as the chopped-random basis quantum optimization. In addition, owing to their reduced number of variables, such bang-bang protocols are very well suited to optimization tasks, reducing the high computational cost of other optimal control protocols. We benchmark the performance of such approach through two paradigmatic models, namely the Landau-Zener and the Lipkin-Meshkov-Glick model. Remarkably, we find that the critical ground state of the latter model, i.e. its ground state at the critical point, can be prepared with a high fidelity in a total evolution time that scales slower than the inverse of the vanishing energy gap.