For follow-up discussion and general commentary on the topic. Comments are sorted chronologically.
Circumcision was the sign of the old covenant - Genesis 17:10-14. Everyone who was a partaker in the old covenant was to be circumcised. However, God also spoke about circumcision of the heart in both the old and new testaments- Jeremiah 4:4, Rom 2:29. This circumcision refers to having a pure heart, set aside for God. It was the intention of God for people to have this kind of circumcision as an indication that they were His followers. Later on, the sign of the new covenant which would follow was established, which is baptism.