here is a good explanation on this. and on a side note "x264" files are usually Blu-Ray rips with an extension of any of the listed below.
x264 is an open source, free, h.264 / AVC video encoder. It is one of the most advanced publicly available tools for encoding h.264 video. It is also a High Profile AVC encoder, one of very few publicly available. Although it is not an actual format and instead an encoder, many files will use the term "x264" in the filename so it is important to understand how to play files that are encoded with x264.
Better understanding x264
Ok so now that we have seen that the term x264 refers actually to an encoder and not a format such as say MKV we should get a better understanding of what type of formats are actually contained when a file refers to x264.
The most popular containers for AVC video are at this point are MKV, MP4, OGM and MOV and when you have a file that says x264 in the filename you will have one of those 4 previously mentioned formats as the video format.
You must distinguish between Video Formats and Encoders/Decoders (Codecs) here!
For example MPEG-4 Visual (Part-2) is a video format. It's defined in the MPEG-4 standard. That is documents and specification, but not a software you can use. Encoders/Decoders (Codecs) are software. They implement standards. Both, DivX and Xvid, are MPEG-4 Visual (Part-2) encoders. So both of them implement the MPEG-4 standard. Internally they may use compeltely different algorithms (or pretty similar ones), but at the end they produce a valid MPEG-4 Visual (Part-2) bitstream that can be decoded by any MPEG-4 Visual (Part-2) decoder. That's because a video compression standard ("Format") only defines how a valid bitstream has to look like. It doesn't tell you how to transform an uncompressed video into such a bitstream. That's up to the Codec developers to figure out...
And the same goes for H.264. While H.264 is a video format (or "standard"), it is not a Codec. x264 is a Codec (actually it's only an encoder, not a decoder) that implements the H.264 standard in software. There are many other H.264 encoders, such as Ateme, MainConcept and so on. All of them produce a valid H.264 bistream at the end (hopefully ^^), but not all of them deliver the same quality. Some encoders use more sophisticated algorithms than others and thus produce better quality. For example x264 beats most (all?) other H.264 encoders at the moment...