See this New Scientist article. Excerpt:

When the group looked at each individual brain scan, however, they found that very few people had all of the brain features they might be expected to have, based on their sex. Across the sample, between 0 and 8 per cent of people had “all-male” or “all-female” brains, depending on the definition. “Most people are in the middle,” says Joel.

This means that, averaged across many people, sex differences in brain structure do exist, but an individual brain is likely to be just that: individual, with a mix of features. “There are not two types of brain,” says Joel.