Explanation and brief history

These file extensions are new container formats. A container, as its name suggests, holds the information in the media file (audio streams, video, subtitles, etc). As an avi file could be encoded with XviD, DivX, or x264, so can mp4 and mkv. This is much more than a name change however, as mp4 and mkv are a huge step up from avi, allowing you to contain entire movies, with menus, subtitles, multiple audio streams, and more onto a single file.

Avi is a very old container format, last updated in 1996. Microsoft played a role in avi's near immortality; their Windows Media Player has limited support of the new formats. Like Internet Explorer, WMP comes preinstalled with Windows and is utilized as the default media player for many people. It has lasted this long through hacks that let it have, to some extent, new features such as subtitles or chapters. This is the reason why containers like mp4 and mkv were created.

What are they?

MP4

Real name: Mpeg 4 part 14
Creator: Mpeg group

Common File extensions
Audio: mp4, m4a, m4p
Video: mpv, mp4v, mp4

Mp4 is a multimedia container format, created as a part of the mpeg-4 standard. Based off Apple's Quicktime container format (mov), it can be used to store images, subtitles, video and audio. It is more popular than mkv due to Apple's use of it in their iTunes music store for both music and videos. The official file extension is mp4, so people generally refer to it as mp4.

MKV / MKA

Real name: Matroska
Creator: Community effort, dozens of developers

Common File extensions
Audio: mka
Video: mkv

Mkv is a multimedia container format like mp4, however it was not created as a standard. Intended as a flexible, extensible alternative to formats like mp4 of asf, it was created with freedom in mind. The specs of mkv are public domain, so anyone may use or modify them as they see fit. Mkv is the most common format Matroska is seen in so its simply called mkv.

MKV vs MP4

Mp4 enjoys much more software / hardware support, being an mpeg group standard and used by Apple. However mp4 does not support as many features as mkv. While the differences are minor, mp4 has a few limitations on what you can use in it, while mkv does not.

Watching them

Support for these advanced formats is sadly limited. It isn't a codec issue; media players have to support the container format in order to play them. I recommend VLC since it can play both file formats with few exceptions. Since no extra codecs are required, VLC is also the easiest option.

Someone on the Gotfrag forums posted an interesting program that combines several media programs into a single installer, known as the Combined Community Codec Pack or CCCP. It uses Media Player Classic or Zoomplayer as the media viewer, along with ffdshow, Haali Media splitter, and a few others to give CCCP the ability to play mkv and mp4 files.

Downloads
VLC Player
Combined Community Codec Pack