The SS Barracks were built on the western outskirts of the Party Rally Grounds between 1936 and 1939, to a design by architect Franz Ruff. They were among the largest barracks built by the National Socialists, and were intended for SS units. The entire complex consisted of the central main building with a "Portal of Honor"; two side wings, each surrounding a courtyard; and several additional buildings.

The U.S. Army captured and occupied the SS Barracks on April 18. 1945. Until the end of 1946, they ran a displaced person camp here, housing several thousand people. After the camp was closed, the Army renamed the buildings the "Merrell Barracks" and kept them in use until 1992 to house the soldiers of the 2nd Armored Cavalry Regiment.

During World War II, the barracks served as a training post for radio operators. The military building also became one of the outposts of the Dachau and Flossenbürg concentration camps. About 150 inmates were deported to the SS Barracks and used in a variety of jobs – including work for commercial companies and offices of the city administration.
 
 

Since 1996, the refurbished barracks have been the headquarters of the Federal Office for Migration and Refugees.