He does, however, recognize the phenomenon the show has become, airing in countries across the globe.
"It's a show that means a lot to a lot of people, and, you know, we've obviously gone into a lot of homes on a Sunday night, so people feel very close to the show, which is great," Stevens said.
On "Downton," the relationship between Crawley and Michelle Dockery's Lady Mary was a central storyline.
Stevens left the show to pursue other opportunities, which have opened up for him. He recently starred in "The Heiress" on Broadway with Jessica Chastain and shot a role in an upcoming film about the founder of WikiLeaks with fellow British actor Benedict Cumberbatch.
Stevens is now filming a movie in Brooklyn with Liam Neeson called "A Walk Among the Tombstones." That part has required the 30-year-old to trim down a bit and darken his hair. When he was first spotted publicly last weekend at the GLAAD Awards sporting the new look, photos quickly spread online with chatter about the change.
Stevens seemed especially surprised — if not confused — by the interest in his appearance.
"Really? OK, I suppose," he said. "There's so much else going on in the world."
He was more interested in encouraging people to visit his homeland.
"It's a great country. There's all sorts to offer. ...There's culture, music, art, food," he said. "The food is much better than it used to be, and the weather is getting better, too! "