When Richard Rouse first came to seek his fortune as a free settler in 1801, he prospered quickly. Using his shrewd initiative and a bit of luck he weathered droughts and depressions to amass 230,000 acres of property by the time he died in 1852. His descendants transformed Rouse Hill House into a fashionable Victorian home, and for two generations enjoyed affluent lives filled with picnic races and parties. But luck began to run out with the 1890s depression which, coupled with a lack of business sense started the family on a downward spiral.

The tragic death of Kathleen Rouse in China in 1932 added to the drama and slowly the land was sold or subdivided. Life continued at Rouse Hill until 1993 with the house seemingly frozen in time. When Gerald Terry, the last family member to occupy the house moved out, only 20 acres of the original 450-acre grant remained.