In structural credit risk models, default events and the ensuing losses are both derived from asset values at maturity. Hence, it is of the utmost importance to choose a distribution for these asset values that is in accordance with empirical data. At the same time, it is desirable to still preserve some analytical tractability. We achieve both goals by putting forward an ensemble approach for asset correlations. Consistent with the data, we view them as fluctuating quantities for which we may choose the average correlation as homogeneous. Thereby, we can reduce the number of parameters to two, the average correlation between assets and the strength of the fluctuations around this average value. Yet, the resulting asset value distribution describes the empirical data well. This allows us to derive the distribution of credit portfolio losses. With Monte Carlo simulations for the value-at-risk and expected tail loss, we validate the assumptions of our approach and demonstrate the necessity of taking fluctuating correlations into account.