Penn Badgley struggled with the "extremely unnatural" elements of fame and even considered giving up acting at one point.