Puritans did not believe in the church of England. They found the Catholic practices the church clung to despite not being Catholic to be repugnant. They came to America to get away from church practices such as giving indulgences to the rich. They wanted to purify their religious practices.
