I used to hate facial hair, but now it's starting to grow on me.
Why this joke works
When an Opinion Grows Literally
Something that “grows on” a person becomes more likable over time. Facial hair also grows on a person's face in the plain physical sense. The opening sounds like an ordinary change of opinion, while the last three words reveal that the speaker may simply be describing a beard. Deliver it casually and avoid emphasizing the pun too early. It is strongest when listeners briefly believe they are hearing a sincere confession before the literal meaning appears.
A closer look
A Change of Heart Becomes a Change of Face
“I used to hate” establishes a strong old opinion, so the softer phrase “starting to” suggests gradual personal growth. The listener expects a reflection about changing taste until the facial-hair subject makes the final expression visible on the speaker’s face. Keeping the sentence in first person helps both readings coexist: an attitude may be improving while a beard is physically appearing.
Fact behind the joke
Hormones Affect Beard and Scalp Follicles Differently
A PubMed-indexed review describes an “androgen paradox”: androgens stimulate beard growth but can suppress growth in androgenetic scalp hair loss. Hair follicles in different body regions respond differently to the same class of hormones. Facial hair growing on someone is therefore part of a location-specific biological response, not simply the same process occurring everywhere on the body.