Islam Feruz gave the visitors a lead they held for more than an hour after 13 minutes, before Olivier Ntcham deservedly levelled the score 14 minutes from time.
Both teams had chances to win it in the dying embers but an additional 30 minutes of extra-time were required to separate the sides.
City had the best of it against a wearying Chelsea side but, despite creating a number of good chances, they just couldn’t find the goal to avoid the penalty shootout.
In truth, neither side really deserved to lose and this encounter would have been a fitting final but Chelsea progress after winning the shootout 5-4 in sudden death.
A top four finish was City’s target going into the season, meaning anything from this semi-final clash with Chelsea would have represented a significant bonus.
Liverpool or Manchester United would await in next week’s final – high stakes for the side who could cope with the occasion best and perform to the high standards both teams have operated at all season long.
This was the sixth time the Elite Development Squad had played at the first-team stadium over the course of the campaign and they boasted an enviable record from these games.
The 2-1 defeat to Southampton was the only blot on a perfect record, with wins racked up on the other four occasions.
There was just one change in defence to the team that booked this play-off berth with the 2-0 win over Aston Villa on Sunday.
It was good news for Vieira, as vice-captain Shay Facey won his fitness race to partner Ellis Plummer at the heart of the back four after missing the weekend’s game.
Supporters’ appetites for under-21 football have been insatiable this season, with attendances reaching up to 5,000 for some of the decisive games the Blues have been involved in of late.
There was twice the reason for fans to show up for this encounter as the club had pledged to donate all proceeds from the game to #StephensStory, the inspiring campaign raising money for the Teenage Cancer Trust.
The entire first-team took time out of their Goodison Park preparations to join 4,000 supporters inside the Etihad for a match that had to be settled on the night.
Chelsea had grabbed the early initiative with an eighth minute goal when the teams met in the regular season at Stamford Bridge in their 2-1 win in October and they got off to another dream start in Manchester.
City had begun on the front foot and may have taken the lead when Jordy Hiwula raced through on goal but Blackman made a fine save and it wasn’t long until the visitors capitalised, taking the lead on their very first attack.
...City EDS v Chelsea...
Lewis Baker was clear on goal when he shimmied around Ian Lawlor in the City goal and had the presence of mind to turn and clip the ball back where it came from for Islam Feruz to nod it into the net.
The visitors had a spell five minutes after the goal where they could have extended their lead, especially when Isaiah Brown sidefooted wide with an open goal after the ball bobbled off Plummer’s knee, but City soon recovered and started fashioning chances again.
Most of the best work was coming from the right flank courtesy of the industrious, fleet-footed pair, Adam Drury and Marcos Lopes, who were linking up to great effect and drilling in crosses which caused the away defence to creak.
On three occasions Drury put in surgical centres which dissected the Chelsea backline but the ball just wouldn’t break for Hiwula or Ntcham to slam home from close range.
City’s domination of territory and possession was not passing for passing sake, they were penetrative and creative in their use of the ball but it just wouldn’t break to the forwards and the Chelsea threat was always present on the counter.
This pattern continued on into the second period when Lopes was unfortunate to see his shot saved by Blackman five minutes after the restart when he was played through on goal.
Just as the game looked to be getting away, City under-18s’ top-scorer Thierry Ambrose made another telling contribution from the bench after his goalscoring exploits at the Etihad last time out.
Ten minutes after coming on, the French youngster pressed and pickpocketed the last Chelsea defender and squared for his compatriot Ntcham to swivel and fire home the equaliser 14 minutes from time.
That’s how it stayed until the final whistle, meaning an extra half an hour was required and, although both teams looked tired after long, draining seasons across four competitions, City looked to have slightly more left in the tank.
With the game stretched, Cole had a couple of shooting opportunities but missed the target and Ambrose appeared to be felled in the area in the last minute of the first half by the goalkeeper but he was booked for simulation.
Penalties felt inevitable as limbs continue to weary and, when they came, they brought about their usual blend of heartstopping drama, ecstasy for one side and desolation for the other.
Fofana’s spot kick was saved by Blackman, giving Baker the chance to win it for the visitors but Lawlor made a brilliant save to take it to sudden death.
The last player who deserved to miss the crucial penalty was Ntcham after a brilliant performance in the heart of midfield but fate turned its back on the Frenchman as the Chelsea keeper made a fingertip save to deny him, handing Ruben Loftus-Cheek the chance to win it.
Unfortunately for the home side, it was an opportunity the England youth international seized definitively.
Despite this set-back, the EDS can look back on an incredibly positive, exciting first league campaign under Vieira and look to build on their successes in 2014/15.
They still have one final chance for silverware this year, with a Manchester Senior Cup final against United coming up soon.
You can donate to Stephen’s Story by visiting his Just Giving page here or you can donate £5 by texting STEPHEN to 70500.
Keep up to date with Stephen’s progress on Twitter at @_StephensStory.
