Born in 1695, Francis Wise was the son of a mercer in Broad Street, attended Trinity College and was then appointed sub-librarian at the Bodleian. As this post only paid £10 a year, he was obliged to supplement his income by giving tutorials. Fortunately for him, one of his first pupils was Francis North, son of the Earl of Guildford, and because of his connection to that family, he was given the living of Elsfield. In 1746, he resigned his post at the Bodleian and became Librarian of the newly built Radcliffe library which carried a salary of £150 a year, with little work attached to it. It was said that the library was “little cumbered by books and almost entirely unencumbered by readers”. He was an antiquarian and a keen collector of coins. He left his mark in Elsfield not only by building the house but also by having a series of ‘modern antiquities’ constructed in the grounds, among which featured a ‘Roman’ altar, a water course and temple, and a cottage constructed to look like a medieval chantry. His antiquities were much commented on and, as has already been noted, were visited by Dr Johnson and the indefatigable Boswell in 1754. The’ Roman’ altar and the ‘druidic temple’, rendered even more Gothic by its state of disrepair, remain to this day.
Following the demise of Rev Wise, the house continued to be occupied by his sister, Mrs Margaret Wise, who gave her brother’s collection of coins to the Bodleian, and after her, Dr Oglander and his wife. The Oglander name, however, appears much earlier in the history of the village, written on a 1703 estate map of the parish in the place where the Manor now stands.