The garden, 5.5 acres in size, was once a prune orchard, but was bought by San Jose in 1927 and is now exclusively devoted to roses. It features more than 3,500 shrubs representing 189 rose varieties.
As an official Display Garden for the All-America Rose Selections (AARS), the San Jose Municipal Rose Garden receives AARS award-winning roses in advance of public release. In the early 2000s, city budget cuts impacted the garden, which led to the AARS placing it on probation. In 2007 Friends of the San Jose Rose Garden (FSJRG) was established, a volunteer nonprofit organization whose mission was to restore and renovate the garden. In 2008 the AARS lifted the garden from probation and restored full accreditation, and the following year over 800 new roses were planted as part of the restoration project.
On May 2, 2009 it was rededicated as an official AARS test garden, one of 23 in the US and the only one in Northern California.
On December 15, 2009 the restoration of the San Jose Municipal Rose Garden was nationally recognized on the Corporation for National and Community Service website.
On August 12, 2010 the AARS named it "America's Best Rose Garden", in the first nationwide contest of its kind. The winning garden was selected based on beauty, creativity and community involvement.
The Municipal Rose Garden is a City of San Jose historic landmark.