While incarcerated for killing his wive in an act of jealousy, he embarks on a "psychogenic fugue" as an act of last-minute escapism from the looming dread of his upcoming execution--sort of like Ambrose Bierce's "Occurence At Owl Creek Bridge"--imagining himself as a younger, more likable/worthwhile guy (valued auto mechanic, "Mr. Eddy's" favorite), with people who care about him (his parents and girlfriend, as opposed to his real-life murdered wife who didn't even bother to go to his musical performances), and definitely more virile, as he is able to both attract and fulfill his "wife" (seen here as the slutty, icy femme fatale-type he always suspected her to be). However, try as he may, he ultimately can't avoid his past (notice how the fantasy him is put off when he hears Fred's jazz song on the radio in the garage), and thus after the fantasy Alice/Renee rejects him in the desert, he immediately turns back into his typical view of himself--hurt, older, sensitive, vulnerable (represented by his nakedness)--proving that even his fantasies fail him, and thus he's left to die an unpleasant death in the electric chair after all (notice the way he violently contorts in the closing moments, almost as if he's being electrocuted). Call him a modern-day murderous Walter Mitty I guess. The Fred Madison/O.J. Simpson comparisons made by some are interesting--if just a BIT cynical!--though I have to halfway wonder if that real-life spousal jealousy murder case provided any grain of inspiration for this fictional one. The cast is impressive and do a great job; Bill Pullman definitely has the haunted, deer-in-the-headlights look that his confused, out-of-it character requires, though at the same time I don't know if he quite portrays the extreme jealousy and animal savageness deep down inside that caused him to murder his wife as gruesomely as he did (if of course you even want to accept what was on that final videotape as something that actually happened in the first place!). Needless to say, the whole moebius-strip "twist" of having the film end at its beginning greatly complicates any interpretation; even without it, the film could STILL be difficult to decipher by some (heck, I'm still not even really sure what the significance of the Mystery Man was!)
Perhaps the film could have benefited from a few extra scenes or lines of dialogue to make it a little less cryptic for the more literal-minded members of the audience, but still, even by suggesting that you'd be implying that there was one concrete explanation for the film, which there most certainly is not.
Regardless, all plot and interpretations aside, you can almost certainly enjoy for its images, its music (an EXCELLENT soundtrack), for its mood and atmosphere, and simply for it as a whole: dare I say, it's almost more of an experience than anything (though for what it's worth, at the same time I can't think of the last time I saw a film--or work of art period, for that matter--that provoked such a wide variety of interpretations and opinions, as should hopefully be the case with ANY great work of art).