For follow-up discussion and general commentary on the topic. Comments are sorted chronologically.
Christian Knickerbocker
I have heard it said that the prophet was Moses, simply by dint of the fact that they called him THE prophet. I don't think it’s right. I don't believe it was Christ, either, because further along in the conversation the pharisees ask, "Why then are you baptizing if you are neither the Christ, nor Elijah, nor the prophet?" So you see that the prophet and Christ were two different figures in their minds.
As for who I think the prophet was, I believe they were referring to what Moses said, "God will raise up for you a prophet like me." I think that at the time the Jews viewed the prophet Moses foretold as a different person than the Christ; thus why they listed three people in questioning John (the prophet, Elijah, and Christ).