I happened to reread an article from the Wall Street Journal of January 31, 2010 today along with the letters to the editor in response to the article.  Here are a couple of interesting excerpts

High-speed rail does little to unsnarl traffic jams because most highway congestion is within urban areas, not between them. It also has negligible impact on airport congestion. The world's strongest high-speed rail market, Tokyo to Osaka, is also one of the world's largest airline markets. Even with high-speed rail, there is still frequent air-shuttle service between Paris and Marseille.

Environmental claims are misleading. Using California High Speed Rail Authority's data, Joe Vranich and I estimated that the California system would reduce the emissions of greenhouse gases, such as CO2, at a cost of $2,000 per ton. The Intergovernmental Panel on Climate Change estimates that we should be able to meet its greenhouse gas targets by spending $50 or less per ton.

A letter writer from Scottsdale, AZ adds some further insight based on his understanding of the Japanese Shinkansen "bullet train":

You don't just lay rails, buy the Very Fast Train (VFT) equipment and then run it. Rails for any VFT must be kept perfect—perfectly aligned, perfectly level, perfectly clean, perfectly clear. In Japan there are two rail maintenance workers per mile on that dedicated right-of-way. I cannot really see Amtrak or the states of California or Florida trying to add that labor cost to their budgets.

The last comment is interesting because a lot of the focus has been on the cost to build the tracks and stations, but I haven't seen as much about the cost to maintain a precise set of tracks.