For many, the stark structure built by the Japanese and then taken over by South Korea's military is a reminder of a painful colonial past and the torture allegedly overseen there later during decades of authoritarian governments.
But rather than bulldozing the Defense Security Command building, South Korea's capital is trying to make peace with its difficult history by transforming the building into a branch of the National Museum of Contemporary Art.
On Wednesday, the culture minister, Choung Byoung-gug, cut the tape at a groundbreaking ceremony for the museum just east of Gyeongbok Palace. Construction is slated to finish by the end of 2012.
Not everyone is happy.
Kim Byung-jin, 56, says turning the building known as Kimusa into a museum makes light of its violent past. He recalled how in 1983, South Korean interrogators in the building stripped off his clothes, beat him to near-unconsciousness and left him on the cold concrete floor as punishment for fainting after four days of sleep deprivation.
"The site was the epicenter of human rights violations after our country's independence," he told The Associated Press, his voice shaking with rage. "Transforming the building into an art museum is an act of appeasement. I am completely against it."
Some want city officials to demolish the building, as they did in the mid-1990s with the neoclassical colonial headquarters the Japanese built in front of Gyeongbok Palace during their 1910-1945 occupation of Korea.
But after years of debate, officials named the Kimusa building a national cultural property in 2008. President Lee Myung-bak announced plans to transform it into a museum the following year.
In a city with breakneck development, where old neighborhoods routinely are demolished and replaced with glass and steel high rises, there's been a recent push to preserve Seoul's remaining pockets of history.
In 2002, a courthouse built by the Japanese in 1928 and then used by the South Koreans to house the Supreme Court reopened after extensive restoration as the Seoul Museum of Art.
Crumbling old Seoul Station has also been restored for art exhibitions and performances, following in the footsteps of cities like Berlin and London that have turned empty historical buildings into art spaces.
"It's not just a train station, but the face and gate of Seoul," said Chung Jae-jeong, president of the Northeast Asian History Foundation.
The station, built by the Japanese in 1925, had been closed since Seoul built a new high-speed rail station in 2004.
The Japanese built the three-story Kimusa building in 1928 to serve as an Imperial Army hospital. It occupies a prime location in Seoul, across from the largest of its royal palaces and along a stretch in the trendy Samcheong-dong area that houses some of the city's top art galleries.
The South Korean military took the building over in the 1970s, allegedly torturing scores of people, including students and activists, within its walls during the years of authoritarian rule.
Kim said he was a literature student at Yonsei University when officials accused him of being a communist spy. Detained, interrogated and tortured for three months, he said he confessed to the "nonsensical" charges after agents threatened to send his infant son to an orphanage and his wife into prostitution.
Military interrogators strapped him into a chair, bound his arms and legs and tied electric wires around his fingers, he said.
"The guards then began shooting continuous bolts of electric shock through my entire body," Kim said by telephone from his home in Osaka, Japan. "The electricity pelted through my brain."
South Korea's Defense Ministry denies accusations of brutality, saying torture never took place in the Kimusa building, without elaborating.
There were only hints of the building's alleged violent past during Wednesday's ceremony. Dignitaries spoke of creating a world-class museum and a new Seoul landmark _ but also of the building's spiritual transformation.
"I would like to express my gratitude and joy as an artist and a citizen in seeing that this site is not being forgotten and buried away with historical traces and scars, but is being reborn as a space of culture and communication," artist Moon Kyung-won said.
Architects plan to preserve part of the building while creating an entirely new complex.
Barry Bergdoll, chief curator of architecture and design at the Museum of Modern Art in New York and a member of the panel that chose the winning bid for the building's design, called it a brave decision to use a place "contaminated with the memory of political experiences from a period of dictatorship as a museum of contemporary art."
At least one professor of architectural history objects to the plans.
Prof. Kim Jeong-dong of Mokwon University in Daejeon said a modern art museum will clash with Gyeongbok Palace and bring congestion and noise "to an area that should be treated with respect, as it once housed our royal family."
Relics found during the excavation will be kept on the museum grounds, officials said.
"It is a place of terrible tragedy," said museum chief Bae Soon-hoon. "However, to transform the pain and turn it into a place where people can come and enjoy _ that's a wonderful thing."
Associated Press writer So Yeon Kwon contributed to this report.