Gradishar is a 1970 graduate of Champion High School, Champion, Ohio. During his high school career, Randy lettered all three years in both football and basketball. As a high school football player, Randy received honors including All-League, All-County, and the Star Helmet Award.

Gradishar, who graduated with a degree in Distributive Education, was a three-year starter with the Ohio State University from 1971 to 1973. Former Ohio State head coach Woody Hayes called Gradishar "the best linebacker I ever coached". He made 134 tackles in his senior year, 60 of them solo, to lead the team.

Gradishar was a consensus First-team All-America selection in 1972 and a unanimous First-team selection in 1973. Also in 1973 Gradishar finished sixth in the voting for the Heisman Trophy.

Randy was also academic All-American in 1973. According to scouts Gradishar was a linebacker who went out every day with his hard hat and lunch pail and got the job done. When naming Gradishar All-American Time stated he was the "best Big Ten linebacker in three years" Gradishar is a punishing tackler capable of penetrating any block, and, say the scouts, "he has that great pro quality—the ability to cover somebody else's mistake." Before entering the National Football League Gradishar played in the Hula Bowl and the Coaches' All-American game in 1974.

Gradishar was drafted 14th overall in the 1974 NFL Draft by the Denver Broncos. Gradishar went on to spend 10 seasons with the Denver Broncos franchise. He played along with Tom Jackson as part of the Orange Crush Defense and is considered by some to be the greatest defensive player in Broncos history. Teammate Jim Jensen said Gradishar and Roger Staubach were the two best players he ever took the field with.

Gradishar became nationally known in a 1975 Monday Night Football game versus the Green Bay Packers in which he picked off a pass and returned it for a game-sealing victory while also recording a sack, two tackles for loss and knocked down two passes and recorded six solo tackles.

In 1976 the Broncos switched to a 3-4 defense in which Gradishar moved from middle linebacker to inside linebacker, where he stayed the remainder of his career. The switch to the 3-4 defense gave Gradishar responsibilities relative to rush-lane discipline and pass coverage that are beyond those of a middle linebacker in the 4-3 defense. In addition to leading the Broncos in tackles for the second straight year, Gradishar led all NFL linebackers in sacks, with seven. He was voted second-team All-AFC by United Press International.

In 1977 Gradishar assisted the Broncos to Super Bowl XII, their very first. He earned Pro Bowl honors and was named first team All-Pro. Gradishar anchored a defense that led the AFC in fewest points allowed (148 in the 14-game season) and gave up the 6th fewest yards. Gradishar was also voted the AFC Defensive Player of the Year by the Columbus Touchdown Club. The 1977 Broncos season is documented in a new book by Terry Frei, '77: Denver, the Broncos, and a Coming of Age that features Gradishar, Lyle Alzado and others. The 1977 Broncos season began with a 7–0 shutout of the St. Louis Cardinals that had been one of the NFL's top offenses from 1974–1976 that featured offensive stars like Jim Hart, Terry Metcalf, Mel Gray, Dan Dierdorf and Conrad Dobler. In a 14–7 win against the Kansas City Chiefs the Broncos made a goal line stand as time expired to preserve a win. Denver Post staff Writer Terry Frei's book, 77: Denver, the Broncos, and a Coming of Age, put it this way, "Gradishar penetrated and stopped two running plays, as Gradishar did time and time again on the goal line in his career. It was the trademark of both Gradishar and the defense overall".

He was named consensus NFL Defensive Player of the Year in 1978 by winning the Associated Press Award, along with the UPI Defensive Player of the Year award and the NEA Defensive MVP award, receiving the George S. Halas Trophy for his efforts as the Broncos reached the playoffs for the second straight season. The Broncos defense allowed 198 points in 16 games, which was second in the NFL behind the Super Bowl champion Steelers. Additionally, Gradishar was the Football Digest NFL Linebacker of the year in 1978, an honor he repeated in 1979.

The following season, 1979, the Broncos defense ranked 5th in fewest points allowed and Gradishar led the team in tackles for the 5th straight season. The 1980 season was average in terms of record (8–8) and statistics as the Broncos defense ranked in the middle of the pack in many defensive categories. Against the Cleveland Browns at Cleveland Browns Stadium, Gradishar returned an interception 93 yards for a touchdown which was his career long interception return, a team record, and one of four defensive touchdowns Gradishar scored in his career. He was also named All-Pro and was voted the Broncos Most Valuable Player for the second time in three years.

Zander Hollander, who wrote The Complete Handbook of Pro Football, said about Gradishar, "Not as nasty as Butkus, doesn't snarl like Lambert, but Gradishar makes all the plays. Hey, he can hit, too. Just ask Saints' Henry Childs, after Gradishar knocked his helmet off, "It was the first in my career like that, in the open field". Gradishar's hitting ability was a sentiment echoed in Rick Korch's book, The Truly Great. In it, Tony Dorsett recalled the hit Gradishar gave him in a 1980 game, "I ran a pass pattern and was wide open but Danny White did not see me. I go back to the huddle and tell Danny that I am wide open. I ran the same route again but this time I was almost decapitated. My eyes were only partially open when I hit the ground. Trainers and doctors came running onto the field. They thought I was dead. Hey, I thought I was dead, too." Hall of Fame defensive lineman Dan Hampton remembers asking Walter Payton, "Walter, who gave you the hardest hit you ever took in the NFL?". According to Hampton, Payton replied, "Randy Gradishar, 1978". In 1981 SPORT magazine named Gradishar one of the Top 5 hardest hitters in the NFL, quoting the modest Gradishar, "The chance for a real good shot comes very seldom, but when it's there I take full advantage of it".

However, under new coach Dan Reeves the Bronco defense rebounded in 1981, ranking 6th in total defense and 9th in fewest points allowed as Broncos finished the season with a record of 10-6. Randy had one of his best seasons as he was consensus All-Pro and All-AFC as well as being voted to the Pro Bowl.

Gradishar retired after the 1983 season, finishing his career with the NFL record for most tackles all time with 2,049, and 20 interceptions, which he returned for 335 yards and three touchdowns. He also recovered 13 fumbles, returning them for 72 yards and one touchdown. He also recorded 20 sacks according to Bronco team records. Since his retirement, he has been nominated for the Pro Football Hall of Fame several times, but has yet to be enshrined. He was among the 15 finalists in 2003 and 2008, and made the top 25 list in 2005 and 2007. The closest he came to induction was the day before the 2003 Super Bowl, when he made it to list of final ten.

Gradishar was inducted to the Broncos' Ring of Fame in 1989. Inducted into the Colorado Sports Hall of Fame in 1987. The Professional Football Researchers Association named Gradishar to the PRFA Hall of Very Good Class of 2008

The Ohio State end-of-season award for most outstanding linebacker is known as the Randy Gradishar Award. Recently named as the 8th best Ohio State player of all-time. Also made list of the Top 100 college football players of all-time. In 2000 was named to ABC Sports's All-Century team as an inside linebacker.

In 2004 Randy was inducted into the Champion High School Hall of Fame for Athletics. He was presented by his former high school coach, Al Carrino.

When naming the top linbeackers ever, Buchsbaum wrote, concerning Gradishar, "Maybe the smartest and most underrated ever. Had rare instincts, was faster than Lambert and very effective in short-yardage and goal-line situations. The fact he is not in the Hall of Fame is a shame and may be attributed to the fact he was a sure tackler but not a lights-out hitter or look-at-me type of player." In 2006 was named by Riddell author Jonathan Rand as one of the Top 25 linebackers of all-time.

In January 2008, he was voted by a panel of former NFL players and coaches to Pro Football Weekly 's All-Time 3-4 defensive team along with Harry Carson, Lawrence Taylor, Andre Tippett, Howie Long, Lee Roy Selmon, and Curley Culp.

In Gradishar's three years with the Buckeyes, all as a starter, the team had a 25–6–1 record, with two Big Ten Conference championships. Gradishar's final collegiate game was a 42–21 victory over the University of Southern California in the 1974 Rose Bowl, to complete a 10–0–1 season. In his senior season the Ohio State defense allowed only 64 points and posted four shutouts.