The Chronicle reported that the Department of Health and Human Services issued a memo last week calling for employees to "maximize the use of telework," due to conditions around the building.
Former House Speaker Nancy Pelosi has also reportedly expressed safety concerns about her staff who work there.
The plaza has been known for years as a place where drug use and sales are common.
This summer, Governor Gavin Newsom announced he would double the amount of California Highway Patrol officers deployed in San Francisco as part of a plan to crackdown on the ongoing deadly fentanyl crisis.
In May, a rare open-air Board of Supervisors meeting was held outdoors at Civic Center Plaza. Some supervisors pressed the mayor about the persistent drug problems in parts of the city, but the meeting quickly devolved into heckling and a brick was thrown by someone before the meeting had to be moved back indoors.
The open-air drug markets, known to be commonplace in neighborhoods such as the Tenderloin, and the crime that has accompanied them have been blamed for impacting popular retail corridors.
In April, Whole Foods closed their flagship San Francisco store about a block away from the SoMa Federal Building. Whole Foods representatives said it was because of employee safety concerns, although Supervisor Matt Dorsey, who represents SoMa and the Tenderloin, characterized the closure as temporary.
A few months later, it was announced that management of San Francisco's Westfield Centre would be transferred to its lender, who would then appoint a receiver to operate the property. They cited a downturn in foot traffic, occupancy and declining sales as the reason for their decision.
When Florida Governor Ron DeSantis used San Francisco as a backdrop for a campaign ad to further tarnish perceptions of San Francisco in national media, Mayor London Breed came to her city's defense, saying the conservative presidential hopeful was playing politics. She added that while he was quick to criticize, he offered no solutions.
The mayor is performing a balancing act of being a cheerleader for San Francisco but has also acknowledged crime in the city as she pushed for a bigger police budget.
In her back and forth with San Francisco supervisors, Breed had said crime is the number one concern in the city and something everyone is talking about.
Despite the onslaught of retail setbacks, one glimmer of economic hope on the horizon was announced this week when IKEA said they'd be opening their first San Francisco store in the Mid-Market corridor later this month.