While City had the bulk of the possession in the first half, Arsenal had the better chances, with big close-season signing Kai Havertz twice spurning opportunities to bag a debut goal as he had two efforts saved by Stefan Ortega.
Pep Guardiola's side showed more intent in the second half, with John Stones having a header saved, but it was substitute Cole Palmer who broke the deadlock with a curling effort from the edge of the box which left Arsenal goalkeeper Aaron Ramsdale beaten.
Despite going a goal down, Arsenal kept pushing and dug deep to bounce back with a stoppage-time equaliser via a deflection and force the penalty shootout - which they ended up winning after Kevin de Bruyne blasted his effort against the crossbar and Ramsdale saved a tame Rodri spotkick.
The win extended City's run of defeats in Community Shield matches, following losses to Liverpool last season and Leicester City in 2021. It was Arsenal's 17th win in the season's curtain-raiser, moving them into second place behind Manchester United in the all-time tally.