Grove was a dissenting clergyman who wrote popular sermons, and lesser known poetry and hymns that appeared in contemporary hymnbooks and were collected in his posthumous Miscellanies in Prose and Verse. He received little income from his profession and lived happily in relative poverty throughout his life.