You need at least one ground unit in the remaining units. If after combat you are left with only air units, you won't capture the city - and your units will be left outside the city.
Sometimes, I think weird turnblocking mechanics (or bugs?) can leave your units next to the city.