Substitutes Aaron Ramsey and Olivier Giroud each scored in the final seven minutes to hand Arsenal a win over the 2016 champions.
GOAL! New boy Alexandre Lacazette heads the Gunners ahead after only two minutes.
"I must tell you I opened the door for him at some stage (to leave) because it's important for him to play". In the end he made a decision to stay and it was one of the great days of the recent weeks for me.
In the 29th minute, Jamie Vardy gave the visitors a 2-1 advantage, scoring from close range after a superb ball into the area by Marc Albrighton, who exploited a blunder by Arsenal's Granit Xhaka.
However, Wenger insists that Giroud will remain in north London beyond the end of the transfer window despite being given the opportunity to move on and seek more regular football elsewhere.
"No. It's a determined no", he said.
"Well if you were a manager and you would be on the bench when you're 3-2 down with 20 minutes to go, and you win 4-3, you're more happy than concerned".
"We had some avoidable goals which we conceded and overall we have to work on that", he said referring to poor marking in the by his team's defence. "We were quite good a year ago on corners and overall I believe that with a bit of work we can get that out of the system", the French coach added.
Leicester boss Craig Shakespeare was also pleased - even if the Foxes headed back to the East Midlands with nothing to show for their evening's endeavour. He adapts slowly to the way we want to play.
Shakespeare was true to his word and started Riyad Mahrez despite the Algeria global asking to leave the King Power Stadium.
Arsenal would hold on to secure an incredible victory in a magnificently entertaining season opener.