H.264 is video encoding method, an alias name for MPEG-4 AVC compression, also known as MPEG-4 Part 10, defined in joint ISO/IC and ITU specification "Information technology — Coding of audio-visual objects — Part 10: Advanced Video Coding".
The specification is freely available here: http://www.itu.int/rec/T-REC-H.264-201304-I/en
x264 is a library implementing video compression and producing H.264 compatible stream.
See also: