Michael Kors, that bastion of jet-set chic—blond hair, perpetual tan, gold-rimmed aviators, and a billion-dollar brand with his name on it—is in fact quite a homebody. In the four decades since he moved to Manhattan from Merrick, Long Island, to attend the Fashion Institute of Technology, the designer has only ever lived downtown. “People are always like, ‘Oh, Michael Kors. The uptown ladies!’ ” he says in the light-filled media room of his and husband Lance Le Pere’s Greenwich Village penthouse. “And the reality is, I’ve always been a downtown boy.”

He launched his business in 1981 out of a modest apartment in Chelsea—the living room accessorized with sewing machines. “There were initials on the fire escape that read hs, so we used to say, ‘It’s High Society!’ ” he deadpans. After that, he moved to West 28th Street, on a block known for its wholesale shops. “We would tell people, ‘We’re between fur and flowers.’ ” (His ability to put a glamorous spin on things started early.) But he’s been a dedicated Villager since 1983: “I’ve moved numerous times, but always within a five-block radius.”