Former Sports Minister has revealed that made what he called a "very strong offer" to take over the after the Games more than three years ago but the club was turned down by Olympic bosses.
Caborn admits that the 2012 board "missed an opportunity" in not striking a deal because it was determined to provide an athletics legacy, one of the key pledges of 2012 chairman Lord Coe during the bid.
Officials from the are currently trying to find a tenant for the stadium and West Ham are still interested in a move there. But rebuilding the arena for top-class football now could cost up to £100 million.
Caborn says West Ham were ready to help change the design of the stadium before it was even built. They made their offer in January 2007, ten months before the design was unveiled.
"West Ham made a very strong offer backed by the Newham Mayor. It was between April 2006 and January 2007," Caborn said. "They wanted to get into negotiations and they did and there was money on the table. They wanted also to honour the athletics legacy.
"Many of us were concerned that that we could end up with a white elephant. I argued very strongly for it but the bid document said there had to be a legacy for athletics, so the negotiations ran into the sand. I don't believe there is a will for a multi-use stadium.
"It was a missed opportunity."
West Ham and are still trying to convince the Legacy Company that they should lease the stadium after reducing its capacity from 80,000 to 60,000 seats. The club would want to move in from the start of the 2014-15 season.
But in addition to adding key hospitality areas, I understand that the roof and the floodlights will have to be scrapped and rebuilt because they only cover a third of the seats.
It is still unclear who would pay for all of these expensive conversions although West Ham are ready to pay towards them.
The 2007 offer, made in a five-page document sent to the 2012 Olympic Board, is said to have been similar to that made by Manchester City when it took over the 2002 Commonwealth Games.
In that agreement, the club agreed to pay cash from ticket sales to fund grass roots sports facilities in Manchester, including the maintenance of an athletics track next to the stadium.