The Criterion Quartet, initially organized by Reinald Werrenrath, recorded for Columbia, Zon-o-phone, Edison, and Leeds & Catlin. This early ensemble included first tenor Robert R. Rainey, second tenor William A. Washburn, baritone Werrenrath, and bass Walter A. Downie. Werrenrath made his debut recording, "Little Tommy Went A Fishing," with the quartet on Edison 8866, released in January 1905. The Zon-o-phone catalog from June 1905 features eight 9-inch records and one 10-inch record attributed to the Criterion Quartet.

By 1916, the group's lineup changed to first tenor John Young (who also served as the manager), second tenor Horatio Rench, baritone George W. Reardon, and bass Donald Chalmers. During this period, the quartet produced numerous Edison Diamond Discs and Blue Amberols, with Harvey Hindermyer and Frank Mellor both contributing as second tenors at different times. Notably, the Diamond Disc "Humming" (50817), released in 1921, marked the final Edison Diamond Disc to feature an etched label, with the subsequent disc, 50818, showcasing Edison's inaugural paper label.

The quartet's Diamond Discs frequently appeared in a classical series, often accompanying renowned artists such as Anna Case, Marie Rappold, and Frieda Hempel on select recordings.

The Criterion Quartet's contributions to Victor records include titles like "Bring Back My Bonnie To Me" (16105), "Old-Time Religion" (16743), "Some Blessed Day" (18808), and "De Coppah Moon" (19042). They also supported featured soloists on various Victor recordings, such as soprano Olive Kline on "Lilly Dale" (Victor 45329) and soprano Hulda Lashanska on "Sweet and Low" (Red Seal Victor 66020), with the quartet credited alongside the soloists.

For Columbia, the quartet recorded "While the Years Rolls By" (A3686; issued November 1922), "Drifting Down" (A3748; issued February 1923), and "It Came Upon A Midnight Clear" (A3998; issued December 1923).

In the early 1920s, Brunswick referred to the quartet as the Strand Quartet, while it was also known as the Criterion Male Quartet. Under its original name, the ensemble achieved success with "Little Cotton Dolly" and "Kentucky Babe" on Brunswick 2228. The same songs were recorded a cappella by the Shannon Quartet for Victor 19013. During the electric recording era, the Criterion Quartet re-recorded "Little Cotton Dolly" (4300), this time with guitar accompaniment. One of the first performances documented using Brunswick's "Light-Ray" electrical recording system was of the quartet performing "Unclouded Day," recorded in July 1925 and released on Brunswick 2931 in October. The group also recorded for other labels, including the Aeolian Company, which produced Vocalion discs.

In the mid-1920s, the Criterion Quartet adopted the name Roxy Quartet, or Roxy Male Quartet, to align itself with the "Roxy's Gang" radio program. This show, hosted by Samuel L. ("Roxy") Rothafel, aired on Sunday evenings over radio station WEAF starting November 19, 1922, coinciding with performances at the Capitol Theatre and later the Roxy Theatre. During this phase, known as the Roxy Quartet, the members included Young, Reardon, Mellor (who joined in 1921), and Fred Thomas, who succeeded Chalmers in the 1930s.