He played high school football at Abraham Clark High School in Roselle, New Jersey, graduating in 1951.

After playing on the defensive line on the Penn State University football team, Grier was drafted as the 31st overall pick in the third round of the 1955 NFL Draft by the New York Giants. He played with the Giants from 1955 to 1962, during which he led the team to a NFL Championship in 1956 and the Eastern Conference Championship in 1958, 1959, 1961 and 1962. Grier was selected for the Pro Bowl in 1956 and 1960, and was named All-Pro at the defensive tackle position in 1956 and 1958–1962.

As a singer, Grier first released singles on the A label in 1960, and over the following twenty-five years he continued to record on various labels including Liberty, Ric, MGM, and A&M. His recording of a tribute to Robert Kennedy, "People Make the World" (written by Bobby Womack), was his only chart single, peaking at No. 128 in 1968.

Sacks with Rams: 1963 (6), 1964 (6.5), 1965 (1.5), 1966 (7).

Grier has appeared in a number of films and television shows. One of the first football stars to successfully make the transition to acting, he made about 70 television guest appearances. They include a role as one of the security contingent in "The Brain Killer Affair" episode of The Man from U.N.C.L.E. (1964), as well as a cameo playing an athletic trainer in an episode of I Dream of Jeannie. He became a regular cast member, starting in 1969, on the series Daniel Boone, Make Room for Granddaddy, and The White Shadow. In one White Shadow appearance, he donned his No. 76 Los Angeles Rams jersey from his NFL playing days.

In December 1968, he accompanied Bob Hope on “Operation Holly,” Hope’s 1968 USO tour, Grier performed alongside headliner Ann-Margret and others at the U.S. bases at Long Bình, Cam Ranh Bay, Da Nang, Chu Lai, and Phù Cát, as well as aboard the carriers USS Hancock and USS New Jersey, and at Korat Royal Thai Air Force Base and U-Tapao Royal Thai Navy Airfield in Thailand, along with stops in South Korea and Guam.

Grier was well known in the 1970s for his hobbies of needlepoint and macrame.

He has been honored by Penn State as recipient of the Distinguished Alumni Award in 1974, and the Alumni Fellow Award in 1991. He was named to the NCAA's "List of the 100 Most Influential Student-Athletes” published to commemorate the NCAA's 100th anniversary. In 1997, he was inducted into the New Jersey Sports Hall of Fame.

Grier has also written a number of books, and now travels the United States as an inspirational speaker. He is a cofounder of American Neighborhood Enterprises, an organization that works to help disadvantaged city dwellers buy homes and receive vocational training. Grier was ordained a Protestant minister in 1983, and the next year he founded his nonprofit resource center for inner-city teens, developing spiritual and educational programs for disadvantaged youths.

Grier is known for his serious pursuit of hobbies not traditionally associated with men. He has authored several books, including Rosey Grier's Needlepoint for Men in 1973. Grier became an ordained Protestant minister in 1983 and travels as an inspirational speaker. He founded American Neighborhood Enterprises, a nonprofit organization that serves inner city youth. He was also a featured speaker at the 1984 Republican National Convention; during its evening session on August 20, 1984, he endorsed President Ronald Reagan for re-election.

Grier released his autobiography Rosey: The Gentle Giant in 1986.

Grier has a daughter from a previous relationship named Sherryl Brown-Tubbs. He later married Bernice Lewis, who had one child, Denise, whom he adopted before getting divorced. He then married Margie Grier and had a son, Roosevelt Kennedy Grier, in 1972. He and Margie divorced in 1978 and remarried in 1980. Margie Grier died on June 10, 2011. He married Wichita school Teacher Cydnee Seyler on April 30, 2013. A nephew, Mike "Big Daddy" Grier, followed his uncle's career in Sports when he enrolled as a student at Boston University, but he played ice hockey instead of football.

Grier was then traded in July 1963 to the Los Angeles Rams in exchange for defensive tackle John LoVetere and a high Future draft pick. He was part of the "Fearsome Foursome", along with Deacon Jones, Merlin Olsen, and Lamar Lundy, considered one of the best defensive lines in football history. His career ended in 1967 due to a torn Achilles tendon. Despite being the oldest member of the Fearsome Foursome, Grier is the last surviving member following the death of Deacon Jones on June 3, 2013.

On January 5, 2017, Grier announced his intention to run for Governor of California as a Republican in the California gubernatorial election, 2018. He ended his candidacy in July 2017.