Interestingly enough, most "Northern" whites were more racist than their "Southern" brethern until the War of Sucession (commonly though incorrectly refferred to as the Civil War) and the Reconstruction. Familiarity is one of the things that helps eliminate racism and prejudice, unless of course one's experiences are mostly negative. Southern whites interacted with blacks more frequently than Northerners. Thomas "Stonewall" Jackson regularly taught literacy to black people at Sunday School, at one point in direct defiance of the law. Robert E. Lee did not believe in slavery. Lincoln did not want free blacks intermingling with whites, he wanted to send them all back to Africa, and men like Fredrick Douglas were a mere novelty to many nothern whites. It was the destruction (war crimes by today's standards) of the South and Reconstruction, along with the way blacks were used by the Union that created the racial hatred that lead to Jim Crow. One example depicted in movies was the "Contraband" in the movie Glory. Most southern whites did not own slaves by the way.