Erbery was born in Glamorganshire. He graduated from Brasenose College, Oxford, England in 1623.

Anthony Wood (1632–1695), the English antiquary, records that Erbery died in London in April 1654 and was buried at either "Ch. Church" or the "Cemiterie joyning to Old Bedlam near London".

He was ejected in 1638 from his Cardiff parish of St Mary's, under the Bishop of Landaff who had branded him a schismatic, after several citations before the Court of High Commission. His offence was refusing, along with fellow Dissenters Walter Craddock and william Wroth, to read the Book of Sports. He became chaplain, when the English Civil War broke out in 1642, to the regiment of Philip Skippon in the Parliamentary Army. According to Christopher Hill

From there he retired to the Isle of Ely. He was a Seeker; in Ely he expanded the Seekers in the 1640s.

He opposed the Baptists, for Example in his 1653 pamphlet A Mad Man's Plea.

He favoured broad religious tolerance, and was dismissive of churches, believing that ‘apostasy’ had set in early in Christian times; and criticized much even in the Independent churches of his time. He attacked the assumption of the sufficiency of scripture, but doubted the Trinity had Biblical support. He believed free grace had been brought forth by John Preston and Richard Sibbes, preached universal redemption, and denied the divinity of Christ. His millennarian views included a Second Coming, but realised by and within 'saints'.