Almost a third of The Avengers may have been set in New York, but as this video from Industrial Light & Magic's YouTube channel shows us, hardly any of the filming took place in the Big Apple itself. Since actually filming in New York would have been difficult—what with the many road-blocking restrictions and the city generally looking down on setting their streets and/or citizens on fire—almost everything had to be digitally recreated.
Using over 250,000 different photographs of New York taken from a variety of heights and at different times of day, the crew was able to replicate roughly 20 square blocks of midtown Manhattan. The live actions shoots, which were mostly held on sound stages in New Mexico, were then placed into this "CGI playground," ensuring a blissfully ignorant, rapt audience.
As incredible as the finished shots themselves are, these behind the scenes glances of actors jumping from enclosed green screens to crumbling cityscapes are equally if not more awe-inspiring.