Fernandinho, Nolito and pulsating game at Celtic Park, which leaves City in second place in Group C on four points after Barcelona came from behind to beat Borussia Monchengladbach.
City conceded goals early in each half and
"In the second-half - the last 20 minutes - we made absolutely everything so I don't know how many chances they had after the three goals, it's a pity because to win here would be a good step forward but we need to (play) from the beginning.
"This group, this Club, we are going to fight until the last game to qualify."
When asked if his side should have won Guardiola declared: "Yes, definitely."
Celtic's goals came via a Moussa Dembele double either side of an unfortunate Sterling own-goal.
Guardiola added: "It's football, it's Europe, the opponents are good. When you see the last 24 games at home for Celtic they lost just three. I know how difficult it is, that's why the reaction was amazing.
"We knew they would start strong, we knew the second goal - it was unlucky - but how we play until the end... I don't remember an action from Celtic after their third goal. Claudio didn't stop anyone else, in the last minutes we had a lot of chances.
"It's okay, it's one point, we didn't lose."