The house that originally stood on the site was built in 1848 and was bought by the Roosevelts in 1854. Theodore Roosevelt was born there on October 27, 1858, and lived in the house with his family until 1872, when the neighborhood began to become more commercial and the family moved uptown to West 57th Street.
Source: Wikipedia