The singer died on Monday in Los Angeles, the family told the Associated Press.
They described him in a statement as a “solitary man with a heart driven to connect to the world”.
“He spoke honestly to people and connected them to each other,” the statement said.
Known for his smooth baritone vocals and sumptuous soul arrangements, he wrote some of the 70s best-remembered songs, including Just The Two Of Us, Lovely Day and Use Me.
On Lovely Day, he set the record for the longest sustained note on a US chart hit, holding a high E note for 18 seconds.
Although he stopped recording in 1985, his songs remained a major influence on R&B and hip-hop.
His track Grandma’s Hands was sampled on Blackstreet’s No Diggity, and Eminem reinterpreted Just The Two Of Us on his hit 1997 Bonnie And Clyde.
Lean On Me has recently become associated with the Coronavirus pandemic, with many people posting their own versions to support health workers.
“We are devastated by the loss of our beloved, devoted husband and father,” said Withers’ family in a statement.
“With his poetry and music, he spoke honestly to people and connected them to each other.
“As private a life as he lived, close to intimate family and friends, his music forever belongs to the world. In this difficult time, we pray his music offers comfort and entertainment as fans hold tight to loved ones.”
US musician Chance the Rapper led tributes, describing the singer as “the greatest” and recalling some of his own personal memories.
Rock star and actor Lenny Kravitz posted that his “voice, songs, and total expression gave us love, hope, and strength”.
Beach Boys mastermind Brian Wilson said he was “very sad to hear about Bill Withers passing” – calling him “a songwriter’s songwriter” – while John Legend added that “life wouldn’t be the same without him” and his music.
And BBC Radio 2’s Trevor Nelson wrote the star’s music was “a remedy for these nonsensical times”.
“He was a genius singer/songwriter. Can’t listen to Bill without feeling emotional.”
He scored another million-selling hit with Lean On Me in 1972.
Gospel-tinged and inspirational, the song was based his experiences growing up in a West Virginia coal mining town: When times were hard, neighbours would lend each other help and assistance, and the memory stuck with the singer.
It was later performed at the inaugurations of both Barack Obama and Bill Clinton.
But Withers quit at the top, walking away from his career after scoring a pop hit with Just The Two Of Us, although he occasionally toured with Grover Washington Jr in the 1990s.
As a younger man, he suffered with a debilitating stutter, and in 2015, he and fellow stutterer Ed Sheeran put on a benefit concert for the Stuttering Association For The Young.
The same year, Withers was inducted into the Rock Hall of Fame, and when asked how it felt by US TV show CBS Good Morning, he joked, “It’s like a pre-obituary!”