Heterogeneity between unemployed and employed individuals matters for optimal fiscal policy. This paper considers the consequences of welfare heterogeneity between these two groups for the determination of optimal capital and labor income taxes in a model with matching frictions in the labor market. In line with a recent finding in the literature, we find that the optimal capital tax is typically non-zero because it is used to indirectly mitigate an externality along the extensive labor margin that arises from search and matching frictions. However, the consideration of heterogeneity makes our result differ in an important way: even for a well-known parameter configuration (the Hosios condition) that typically eliminates this externality, we show that the optimal capital income tax is still non-zero. We also show that labor adjustment along the intensive margin has an important effect on efficiency at the extensive margin, and hence on the optimal capital tax, independent of welfare heterogeneity. Taken together, our results show that these two empirically-relevant features of the labor market can have a quantitatively-important effect on the optimal capital tax.