In 1860, at the height of U.S. slavery, U.S. Census data shows 25% of white Southern households owned slaves, or about 4.9% of the white population in slave states, since only household heads were counted. Nationwide, only 1.4% of whites owned slaves due to the North’s non-slaveholding population.