Rare and fine books. What's the difference? What are antiquarian books? I suppose antiquarian books are what I collect, not really rare. But fine books sounds better.
I started collecting books 25 years ago. I have bought books since my late 20s 35 years ago. But I dipped my toes into collecting in Yucca Valley, CA at Sagebrush Books, buying the complete works of Brett Harte and having them shipped to Brooklyn (where I was living at the time). I've never looked back.
I have an extensive collection of (basically American fiction only, focusing on short stories) American turn of the century fiction (1890s to 1910), 1920s-1940s fiction, 1950s-60s fiction, and 70s and early eighties fiction (Bukowski, etc.).
My sweet spot is the American short story. I have read all of O. Henry's works (for better or for worse, y'all), and not a one of his short stories or poems is bad, because he wrote them. A little time in prison and having to live in smoke-clogged Pittsburgh for 6 months will cause anyone to birth a stellar short story or two.
Where is the best place to buy antiquarian books? abebooks.com, hands down. Can't say enough good things about it. Most dealers worldwide hawk their wares on abe, so you pretty much can find anything you want. This isn't scientific, but I find that I buy a lot of books on abe from booksellers based in CA, PA, NY, VA, MN, MI, and MA.
In terms of walking into stores to buy fine books, I still have San Diego (oddly, maybe) at the top of the list, Royal Books in Baltimore as a knockout punch (holy shit, and holy mackerel their website design rocks!), and the top dog, online only for the most part, across the river from me in South Jersey - Between the Covers.
Baumann (based in Philly) and some other millionaire collection-hunters in NYC and London, probably, are fine and dandy - if you have lots of money. But Between the Covers, with its sheer range of price points and exceedingly rich inventory, holds the BMF belt for booksellers - on the planet!