It is not just about who we pick. He was the heavy hitter of French literature in XIX century, the one who broke with the enlightment names, build himself a huge reputation as poet, as novelist, managed to get popular and was the shadow Flaubert and Balzac had to face and admire. He is the closest thing to a national hero among writers and while we may argue if Flaubert or Verlaine were better at their specific crafts as Hugo (which is purelly an argument, the truth may not be so clear), there is no doubt he is the one who could claim the highest level of quality with prose and poetry unlike any of the other big names (Even Baudelaire, his prose is great, but he rather made it be poetry or essays). Of course, by the end of XIX century, realism made Hugo a bit outdated, his political views naive, but then, you can say the same about Zola/Flaubert when Proust happened.
All in all, Hugo was not the Goethe, Cervantes, Shakespeare of France, just almost, because French literature is rather multiple and they do enjoy cutting heads.