For every lit author like Cormac McCarthy, who borrows science fiction themes, there are ten authors who start out writing science fiction, and then become beloved of literary hipsters. Here's a partial list.
It's funny, when you think about it, that so much attention gets paid to the McCarthys, Atwoods, Lessings and Roths of the world, who are known for their lit writing but try their hand at speculative fiction here and there. There are really just a handful of them who've made an impression, whereas there are tons of SF authors who've gone the other way - and yet, people seldom talk about it.
Mary Shelley, H.G. Wells, Jules Verne and Edgar Allan Poe are all writers whose names come up a lot when you talk about science fiction authors who have become beloved of the lit-mongers. To some extent, I think Shelley, Wells, Verne and Poe predate the genre-ification of SF, since they were writing at a time before the term was invented. In any case, they've all long since been embraced as fully literary. (Well, mostly. Of Poe, critic V.S. Pritchett famously said he was "a second class writer, but a fertilizing exclaimer.") "I get the sense that Wells in his day was a "novel of ideas" guy/polemicist who was later gerrymandered into SF," says Douglas Wolk, author of Reading Comics.
H.P. Lovecraft was clearly a genre writer in his time, thanks to his association with Weird Tales and other magazines. Nowadays, he's the subject of a book-length essay by fancy-pants French guy Michel Houellebecq, and the prestigious Library Of America has collected his stories.
Stanislaw Lem. A few science fiction authors have gotten published in the New Yorker, but Lem was a regular fixture there for years. Lem's satires, in which Ijon Tichy encounters weird time paradoxes, surrealistic societies and philosophical dead-ends, are tailor made for the lit crowd.
Gene Wolfe also got his work published in the New Yorker, as well as other fancy literary magazines. He's been compared to Proust, G.K. Chesterton and Dickens by critics in the Washington Post and other places. (I read a bunch of Wolfe's short stories the other day, and the Dickens comparison seemed particularly apt - they felt almost too 19th century for my taste, but he was clearly doing a good job of capturing a certain smoky industrial revolution feeling to his otherworldly stories.)
Ursula K. LeGuin is another author that nobody even questions the literariness of any more. Her latest book, Lavinia, has been greeted as a pure work of literature, with tons of articles about her contributions to the literature of ideas. The Cleveland Plain Dealer names it one of the best books of 2008, without regard to genre.
John Crowley is the uber-example of someone who crossed over - his earliest novels (The Deep, Beasts and Engine Summer) were pure SF and fantasy. His fourth book, the fantasy novel Little, Big, not only won the World Fantasy Award for best novel, but also praise from mega-critic Harold Bloom. He's since won a American Academy and Institute of Arts and Letters Award in Literature, and he teaches at Yale.
Octavia Butler is another obvious choice. Just look at this gigantic list of journal articles about her, which appears to be outdated in any case. Including things like "Power and Repetition: Philosophies of (Literary) History in Octavia E. Butler's Kindred," from the journal Contemporary Literature.
Philip K. Dick is another one I barely feel the need to justify. Just read this disturbing excerpt from an article in Lingua Franca:
WHEN THE NOVELIST PHILIP K. DICK DIED IN 1982, THE INFLUENTIAL literary theorist Fredric Jameson eulogized him as "the Shakespeare of science fiction." At the time of this encomium, Dick was hardly famous. The author of more than fifty books, he had an enthusiastic following among science fiction fans. But he was rarely read by anyone else.
These days, Dick is far better known. Vintage publishes his fiction in a uniform paperback edition. Hollywood filmmakers transform his stories of imaginary worlds and conspiratorial cartels into movies like Screamers and Total Recall. Meanwhile, academic critics laud him as a postmodernist visionary, a canny prophet of virtual reality, corporate espionage, and the schizoid nature of identity in a digitized world. Indeed, beginning in the last years of his life and continuing to the present, these critics have played a key role in the canonization of Philip K. Dick.
But did Dick return the favor? Not exactly. To their considerable anguish, Dick's academic champions have had to contend with the revelation that their hero wrote letters to the Federal Bureau of Investigation denouncing them. In these letters, Dick claimed that Jameson and other literary theorists were agents of a KGB conspiracy to take over American science fiction. When he sent these messages, Dick was not in the best state of mind: He frequently heard voices and saw visions, often bathed in a mysterious pink light. Even so, the news of his surreptitious campaign against his academic admirers has left some of them deeply disturbed.
Harlan Ellison shows up in a surprising number of college syllabi, including a lot of science fiction writing classes but also a lot of generic "advanced composition" and English classes. His most commonly assigned story seems to be "Repent, Harlequin! Said The Tick-Tock Man." It's assigned often enough that there's a study guide for it. And as his bio proudly notes, he had a story in the 1993 Best American Short Stories.
Samuel R. Delany has easily become a lit-nerd must-read, thanks to his dense, challengiang narratives. It also didn't hurt, notes Wolk, that he's written smut and literary theory as well. "I've quoted him in a theory context," Wolk says. Check out this progression of Delany book covers, from lurid to literary:
William Gibson was heading for literary status for ages, thanks to his cyberpunk classics like Neuromancer and Count Zero. But when he switched to writing books set in the present, with Pattern Recognition and Spook Country, it became fashionable to talk about him as influential in his own time. The New York Times called Spook Country "the first post-post-9/11 novel." He was interviewed in the California Literary Review, and people started talking about the quality of his prose.
Kurt Vonnegut caused some debate among the people I asked about this topic. Was this literary idol ever considered a science fiction author? On the pro side, people cited the ultra-pulpy original covers of Player Piano and Sirens Of Titan, his first two books. "Where do you think [his fictional science fiction author] Kilgore Trout came from?" asks science fiction critic Mike Berry. On the other hand, his early stories appeared in Colliers and the Saturday Evening Post. If anything, one person suggested, Vonnegut is going the other way: He used to have literary cache, but now lit-snobs find him embarrassing. So it goes.
Jonathan Lethem got suggested by several people including Susan Marie Groppi from StrangeHorizons. It took me a while to remember that Lethem actually did start out as a science fiction author, before becoming a literary darling. (Check out his story collection The Wall Of The Sky, The Wall Of The Eye, which contains a bunch of stories that wouldn't be out of place in a typical issue of Asimov's or Analog. And at least one, "Vanilla Dunk," was in Asimov's.)
Ray Bradbury belongs on this list, if only for Fahrenheit 451, which is taught in college lit classes everywhere. Type the term "fahrenheit 451 sparknotes" into Google and you get 9,000 results, thanks to desperate term-paper-writing kids all over. (I didn't even type in that phrase. Google auto-suggested it.)
Weird-cyber author Rudy Rucker pops up on college syllabuses you'd expect, for science fiction writing classes. But also classes in rhetoric, and philosophy.
James Tiptree, Jr. gets on the list, if only because the male persona of writer Alice Sheldon has garnered lots of attention from gender theorists. The recent award-winning biography by Julie Phillips sparked more interest in Tiptree's life in the mainstream media, but also started people paying more attention to Tiptree's mind-bending stories, especially "Houston, Houston, Do You Read."
Neal Stephenson is another author who seems like an obvious inclusion. His latest book, Anathem, was greeted everywhere as a serious novel, not particularly as a science fiction book. Here's Pulitzer Prize-winning book critic Michael Dirda, writing in the Washington Post:
For the past 30 years I've been a zealous advocate for literary science fiction and fantasy, arguing that writers such as Gene Wolfe, Thomas M. Disch, John Crowley, Ursula K. Le Guin, Howard Waldrop and a handful of others are significant American authors, as well as artists of the first rank ... Neal Stephenson has established himself as one of these genre-transcending gods, read passionately by geeks and fans, but also admired as a novelist of ideas, a 21st-century Thomas Pynchon.
(He goes on to say that Anathem is a bit of a disappointment, actually.)
To be honest, I'd forgotten about Thomas M. Disch until I read the above quote, and then smacked my forehead. Before he died, Disch was as well known for his criticism (The Dreams Our Stuff Is Made Of) and poetry as for his fiction, a surefire way to get literary cache. The Telegraph even called him "excessively literary by the standards of his time."
Dan Simmons is in the process of transitioning to literary icon, according to some of my friends who monitor such things zealously. (They have a monitor board, and it lights up when a previously-tagged author starts to swim upstream towards the literary spawning grounds.) His use of literary allusions, especially Keats, probably doesn't hurt. Salon.com proclaimed him a literary master in 2002.
Howard Waldrop is as well known in lit circles as he is in speculative fiction circles. Dirda, once again, champions him as up there with Dick and other postmodern storytellers.
Carol Emshwiller won an NEA grant and a Pushcart Prize, and has had her stories in lit journals like McSweeney's and the Voice Literary Supplement.
Methodology: To some extent, this is just based on years of reading critics, and seeing who actually gets published in literary magazines and stuff. I also did a search on college syllabuses to see which authors are actually getting taught in college lit classes. And I polled my Twitter and Facebook homies.
(I'm leaving out some "urban fantasy" ish writers like Kelly Link, China Mieville or Neil Gaiman, in the interests of keeping this list from being too long. Plus once you get into magic and fantasy elements, you have to talk about the "magical realism" vogue of the 1990s and how that intersected with lit fiction and fantasy. And this is a blog post, not a book.)
Also: I'm not talking here about mainstream success, or having your books made into movies, or becoming a household name. I'm talking about acclaim from lit-nerds, a community that's just as insular as science fiction fans. As a member of both communities, I know they both have their odd grooming rituals and fetishes. An author like Philip K. Dick has long since passed the point of being "cool" for lit-nerds to discover. Now, if you're a lit-nerd and you haven't read Dick, your compadres just look at you pityingly. Ditto for Le Guin or Delany. Nobody in lit circles bothers to call them literary any more, they just are.
Finally, as I said above, this is a partial list. Feel free to suggest other people I left out.