H2exists as the two electrons of H-H bond occupy the lower energy bonding molecular orbital (or sigma bonding molecular orbital), where they are attracted to both positively charged nuclei. It is this electrostatic attraction that gives a covalent bond its strength.
Lack of existence of He2because each He atom comes with two electrons. He2would have 4 electrons, the first 2 filling lower energy molecular orbital and the other 2 filling higher energy anti-bonding molecular orbital. The two electrons in the higher energy orbital would cancel out the effect of lower energy orbital thus He2 cannot exist.