To revisit, a high dynamic range photo combines multiple photos taken at different exposures to create a single picture that looks more like what your eyeballs are able to detect than a standard digital photograph. (Dynamic range is basically the range between the darkest and lightest parts of an image. Check out Ansel Adam's Zone System for more on this.)
HDR photos solve two problems in the iPhone 4: Most digital cameras tend to not have fantastic dynamic range, and the iPhone 4 also lacks manual controls for adjusting exposure beyond tapping the area of the image you want to expose for. This can cause problems in bright outdoor scenes, forcing you to choose between blown out skies or shadowy figures (see above). With an HDR photo, theoretically you'll be able to capture and correctly expose the whole picture, just the way you saw it before you framed it with your iPhone. (Sadly, this is only available for the iPhone 4, not other iOS devices.)
In HDR mode, the iPhone 4 captures three exposures to combine into an HDR photo: underexposed, normal, and overexposed. Even though it's shooting those sequence of pictures pretty fast, it's not instant. So, if you move the phone, or if your subject is on the run, you're going to wind up with some mutant friends with three arms or whispy ghosts when the phone tries to mix all the photos together. As you can see in the picture above, taken while walking, we've got phantom cars, mutant trees and weird road markings.
Having your subject in focus is key to making it look right when the iPhone 4 combines everything into a single image—in part, so it's easier for the software to do its job mixing all of the photos together without scrambling them into a fuzzy, weird mess.
The key is to make sure you have a lot of dynamic range to capture in the first place. In other words, something with a decent range of light and dark areas. Photos that are relatively flat (like in low light), at best, show no improvement or at worst, suffer when you slap HDR on 'em.
Given the iPhone 4's basic HDR capabilities, you'll get the best results with photos where you're trying to do basic things like properly expose somebody's face against a bright outdoor scene. A photo of planks on a boardwalk that's already properly exposed, not so much. (When it's not clear what it should do, the iPhone tends to lean toward an overexposed or washed out look. And with HDR in general, you definitely lose the iPhone's tendency toward pumped contrast and saturation, which I usually prefer.)
The great thing about the iPhone 4's HDR feature is that it preserves the original photo along with the hopefully new-and-improved version, so it doesn't cost you anything to experiment. If you hate the result, just delete it.
The iPhone 4's camera was already awesome. While its new HDR feature doesn't produce miracles, it does make our favorite phone camera even better, so it's hard to complain too much.