The War of 1861-1865 was a war about the Union. Slavery was a side issue. The Emancipation Proclamation was not issue until late 1863 and freed NO slaves in the South.
In the South, most people did not own slaves (look it up). There were black slave owners (again, look it up). Most slaves were owned by large, land owning families that resemble today's corporate farms.
After the invention of the cotton gin provided a cheap, efficient alternative to cleaning cotton seeds out of the cotton bolls, slavery was on its' way to self destruction as it was no longer economically feasible.
The North started "The War of Northen Agression" crushing the South economically and setting race relations back 150 years.