Albania took the lead in the 11th minute as Qazim Laci burst through the middle and headed home from six yards.
However, an inspired second half performance from Zlatko Dalic’s Croatia saw them turn it around with two goals in three minutes.
That looked to be enough to win it, until a dramatic 95th minute equaliser ensured a share of the spoils.
Striker Andre Kramaric netted the equaliser in the 74th minute before an unfortunate Klaus Gjasula own goal two minutes later, following a move started by Gvardiol, completed the turnaround.
Gjasula made amends though, sliding the ball home with just seconds left on the clock to earn an historic point for his country.
Having played as a marauding left-back in Croatia’s opening defeat to Spain, Gvardiol was at the heart of the defence for this game.
That didn’t deter the 22-year-old from stepping out from the back with the ball at his feet, confidently getting numerous attacks under way and completing a total of 60 passes.
He also did all he could at the back, bravely throwing his body in the way late on as Albania pushed for an equaliser.
Meanwhile, Kovacic maintained his relationship with Real Madrid’s Luka Modric and Al Nassr’s Marcelo Brozovic in the centre of midfield.
The No.8 was his usual composed self, rarely flustered on the ball and able to cover a lot of ground out of possession.
His 89 successful passes was the second highest in the game behind Modric, while he completed more tackles than anyone else (four) and had a joint game-high three shots.
It was also our man’s burst forward that opened up the space that Kramaric eventually took advantage of for Croatia’s first.
Our man could even have won it with the very last attack, firing a shot with his left foot that was stopped by Albania goalkeeper Thomas Strakosha.
Croatia now have one point from their two games thus far and their Group B will conclude against Italy in Leipzig on Monday 24 June.