We perform the first-principles calculations to investigate the roles of C vacancy and nitrogen impurity in the magnetic properties of diamond. The coupling is ferromagnetic between the C vacancies in -2e charged state, whereas they prefer to interact antiferromagnetically in -e charged state. Substituting C with N atoms can manipulate the charge states of C vacancies and the magnetic interactions between them. Our work offers a possible route toward high Curie temperature ferromagnetism in metal-free diamond.