But Colombia could not capitalise, and Eric Dier would be the hero for England as he struck his penalty past David Ospina. In many ways, the two teams are too similar, but the evolution under Southgate means that England can at least approach the game with more finesse than in the past.
England have put their fans through misery over the years, but one fans suggested that the Three Lions may go a step further and "kill" him.
"We've talked to the players about them writing their own stories and tonight was the classic, they don't have to conform to what's gone", he said. And every England player knew the order they would shoot in. In goal, Russia's veteran shot-stopper Igor Akinfeev was an obvious choice following his all-important penalty save that sent his team to the quarter-finals of the competition.
"The World Cup is a very special global competition", he said. We started to make a couple of defensive errors and getting a bit edgy.
England became the last team to qualify for the World Cup quarter-finals after a dramatic penalty shootout as favourites Brazil and fancied France trained their sights on their last-eight opponents.
The game was bad-tempered and Geiger at times allowed players to get away with dissent. And, for all the progress they have made under Southgate, their struggle to create chances could yet be the thing that prevents them from winning this World Cup.
"We still have a long way to go in this tournament", Kane, who also scored in the shootout, said as the expectations of England fans rise at the prospect of a less than intimidating match with Sweden and a semi-final against Russian Federation or Croatia. England hadn't won a penalty shootout in 22 years, but this time, it did.
However, England were given hope when Matteus Uribe slammed Colombia's third kick onto the bar and after Kieran Trippier had levelled the score at 3-3, goalkeeper Jordan Pickford then produced a stunning save to keep out Carlos Bacca's fifth penalty for the South Americans.
"They've really bought in to everything we've tried to do".