Incest is rare in the cooperatively breeding acorn woodpecker (Melanerpes formicivorus) despite a polygynandrous mating system in which nearly all group members are close relatives. Here we test the relative importance of avoiding matings between close relatives (incest avoidance) and within-sex competition for breeding opportunities (reproductive competition) in determining the mating system of acorn woodpeckers by examining how reproductive roles change following breeding vacancies. In 83% of cases in which helpers of the same sex were present in the group, reproductive vacancies were resolved when new unrelated immigrants filled the vacancy to the exclusion of resident same-sex helpers, who generally emigrated or did not breed while they remained in the group. Helpers of the opposite sex, especially when male, were significantly more likely to remain in their natal group and in about half the cases inherited and bred following reproductive vacancies. This result was not explainable by reproductive competition, since the number of immigrants was often less than or equal to the number of same-sex helpers in the group. Apparent incest resulted in 5% of cases. The time required to resolve reproductive vacancies was significantly longer for groups with helpers of the same sex as the vacancy. These results confirm that both incest avoidance and reproductive competition are important factors determining reproductive roles within groups of this highly social species.