He was admitted a member of the Order of the Crown of Thorns by Prince-Abbot Joseph III in 1893 and served as Prelate-Commander and its representative in France (later Switzerland) until his death, maintaining a Secretariat at 21 Croix-des-Petits-Champs, Paris. He was formally invested by Prince-Abbot Joseph III in 1898 at 5, Rue Vernier, Paris. His episcopal arms bear a suspension of the jewel of the Order and a photograph of his office shows his commander’s collar together with the sword of his commandery.
He was ordained priest in the Roman Catholic Church on 16 March 1867 by Casimir Wicart, Bishop of Laval, and in 1870 was vicar of Grand Oisseau. At the outbreak of the Franco-Prussian War he volunteered and became a military chaplain. He was acclaimed as a national hero after on a single day rescuing ten wounded soldiers under enemy fire; that same night he led twenty soldiers who had become lost through the forest. After the war he became vicar of Juvigné, but his health broke down as a result of his war service and he was admitted to a military hospital.
He returned as vicar of St Joseph in Paris, where his social and religious views, although popular with his laity, aroused the ire of Cardinal Richard, his ultramontane ordinary, who caused Houssaye to be deposed in 1885.
From this time, Houssaye turned to writing and teaching, issuing periodicals and promoting a Gallican and democratic viewpoint in church affairs. A chance meeting at that time with noted healer Jean Sempé led to an interest in Divine Healing, and Houssaye subsequently made a long study of this subject as well as gaining a reputation as a practitioner. His wide interests embraced much esoteric philosophy including studies in Gnosticism and Martinism, and he was a particular advocate of the use of pentacles. Nevertheless he maintained a strict orthodoxy of belief and worship. During 1901, he met Prince-Abbot Joseph III, and in 1904 founded the Liberal Catholic Church of France, for which he was consecrated by Paulo Miraglia Gulotti (a bishop consecrated by Prince-Abbot Joseph III) on 4 December 1904. Between 1908 and 1912 he served as Primate of the Eglise Catholique Gallicane. His successor was Mgr. Louis-Marie-François Giraud (1876-1951). His relics are preserved in a small chapel in Haute-Savoie.
His books (some published under the pseudonym Benoît Gogo) include: Gorin et Cie, Société d’exploiteurs, Bruxelles 1886; L’Archevêque de Paris et les dames Carreau, Paris 1887; Passibonqueça, histoire véridique et peu surprenante d’un curé de Paris, manuscrit de l’abbé Le Gallo, Paris 1888; Un forçat du bagne clérical, Paris 1888. He was editor of L’Étincelle religieuse libérale, organe de l’Union des Églises and La Tribune populaire, organe de la démocratie religieuse et de la défence du clergé.