Seed comes almost entirely from the female plant ovule. The male gamete, transported in pollen, makes but a minor contribution in catalyzing the process. Hell, many plants form fruit and sometimes even unfertilized yet viable parthenogenetic seeds without male genetic contribution. Seed is a poor metaphor in this instance.