Almost every main character in the story
I don't know if Tom and Daisy can be called drifters. They seemed to fit well with the rich in the East, but they moved because they are rich, careless and never bothered to value others feelings over their personal wealth. Tom in his arrogant, open affair with another man's wife. And Daisy by her shameless admiration for wealth.
Nick and Gatsby are true drifters. Gatsby was never able to truly mingle with the higher society in the East in spite of throwing numerous parties. Nick never felt comfortable with the pompous life style of Daisy, Gatsby, or Jordan.