For theTable of Contents: Berkeley MPEG Tools, [click here]

Background

MPEG is a compression standard for audio, video, and data established by the International Telecommunications Union and International Standards Organization. The MPEG-1 standard, established in 1992, is designed to produce reasonable quality images and sound at low bit rates (e.g., 352x288 images with VHS quality at 1.5 Mbits/sec). See the MPEG-1 FAQ. The MPEG-2 standard, established in 1994, is designed to produce higher quality images at higher bit rates (e.g., 720x480 studio quality CCIR-601 images at up to 15 Mbits/sec). See the MPEG-2 FAQ. For further information see the MPEG Summary which gives a brief introduction to the video standard.

The Berkeley Plateau Multimedia Research Group (now subsumed by the Berkeley Multimedia Research Center (BMRC)) developed the first widely-distributed software decoder for MPEG-1 video in November 1992. The Berkeley decoder has been ported to seemingly every computer system, and it has been distributed to millions of users on the Internet. Our original goal in developing this code was to determine how close the current generation of RISC processors and systems was to being able to decode video in real-time (see paper presented at ACM Multimedia '93). We ran benchmarks for many years and fixed bugs in the decoder, but stopped this work in the late 1990's. Commercial decoders have surpassed the mpeg_play software so they should be used for the best results.

A revised version of the ACM Multimedia '93 paper was recently published in the new ACM Transactions on Multimedia Computing, Communications, and Applications that updated the performance benchmarks using modern processors and discussed the history of MPEG-1 software development and the Berkeley MPEG Research activities. The paper can be downloaded from the ACM Digital Library.

Other decoders of note are:

• The DEC software-only decoder developed in 1993 that was, in our opinion, the first software-only decoder to decode and display MPEG-1 constrained parameter bitrate video streams in real-time.
• The HP software decoder with hardware assist developed in 1994 that was the first system to decode audio and video MPEG-1 streams in real-time. The hardware assist includes parallel execution of 8 bit integer operations and color space conversion (YCrCb -> RGB) in hardware. Both features are standard in HP Risc processors since 1994. A paper was written by Ruby Lee that describes this work.
• The first widely-distributed MPEG-2 video decoder developed by the MPEG Software Simulation Group, an independent MPEG research group, headed by Stefan Eckert and Chad Fogg. Also, a very quick MPEG-1 viewer for MS-DOS and Windows is also available from them.
• An MP3 audio decoder technology

In the spring of 1993, we released a software encoder for MPEG-1 video that runs either sequentially on one computer or in parallel on many computers (see paper presented at PCS '94). The Berkeley encoder has been distributed to over 15,000 users. The encoder and decoder have been widely used in a variety of applications including multimedia research, scientific visualization, and the WWW. In addition, approximately a dozen companies have distributed products based on the code.

The sequential encoder can process approximately .7 fps (CIF) and 3.5 fps (QCIF) on current generation CPU's. The parallel encoder uses frame parallelism which means a complete frame is sent to each processor. We actually encode sequences of frames on each processor to improve locality of reference (i.e., coding frames on the same processor as the reference frames for that frame improves memory performance). Consequently, there is a time delay between receiving a source frame and producing the encoded sequence. Our goal was to build a usable encoder, not a high-preformance real-time encoder. The parallel encoder running on 6 workstations connected by a 10 Mbs ethernet is able to encode approximately 6 times the rate of the sequential encoder. That is, 4.2 fps for CIF and 21 fps for QCIF images, respectively. More details on this performance is given in the paper cited above. In late 1994 students working with Prof. Ed Delp at Purdue modified the encoder to run on an Intel parallel supercomputer. They modified the I/O system and were able to encode over 40 full-sized fps (i.e., CCIR-601 images which are 720x480). A paper describing this work was published at SPIE 95.

We also continue to work on our MPEG Tools. Our recent focus has been the development of perceptual coding models that can produce video streams with the highest perceived quality for a fixed bitrate. To support this work, we have recently been developing a collection of tools for analyzing and hand tweaking the video encoder. More details on this work are given below.

Berkeley MPEG Tools Distribution.

The first release of the combined Berkeley MPEG Tools (Version 1.0) includes:

• mpeg_play A portable software-only MPEG-1 video decoder.
• mpeg_encodeA software-only MPEG-1 video encoder.
• MPEG-1 Video Analyzers:
  • mpeg_stat thoroughly analyzes MPEG streams by analyzing or printing out most of the information in the stream.
  • mpeg_blocks is a visual tool which displays the blocks and motion vectors of an MPEG stream as it is being played via the Tcl/Tk toolkit.
  • mpeg_bits graphically depicts the distribution of bits in the macroblocks of an MPEG stream. This program also uses the Tcl/Tk toolkit.

MPEG/Compression references

• MPEG-1 FAQ
• MPEG-2 FAQ.
• MPEG Pointers page.

Demo MPEGs:

• Various MPEG-Animations (from movies)

What Next?

Many students are looking for research projects involving MPEG video technology. Here are a couple of projects that will be educational and of interest to others:

• Port the analyzer tools to Windows. An excellent port is available for Tcl/Tk, and the MinGW compiler tools should make this a relatively easy task.
• Recent experiments on mpeg_play show that it is possible to decode MPEG-1 videos in real-time. Someone should re-run the experiments on mpeg_encode to update the performance of that code.
• Students interested in learning about multimedia instruction sets, might think about replacing the software DCT and inverse DCT with MMX instructions.
• Another MMX project might be to replace color space conversion with MMX instructions. Many papers about MPEG decoding have shown that color space conversion is a significant percentage of the performance. This code could be improved by replacing the fast ordered dither with MMX instructions.
• The Berkeley decoder cheats, yes, that's right we cheated, by only including two terms when computing blocks using a half-pixel motion vector. You could remove this restriction and time the code with and without the cheat to measure the performance advantage. Our guess is that it would be very small. Moreover, you could compute difference images to see how much difference the cheating had on image quality.
• Build a convenient GUI for running the tools including the encoder. Many people want a simple encoding system, but they also need access to the various underlying coding parameters (e.g., GOP structure, bit rate, etc.).
• Add features to the rate control algorithm in mpeg_encode to improve quality and handling of difficult cases.

While none of these projects is of dramatic interest to the larger video compression research community, they are excellent projects for learning the fundamental ideas in a basic block-transform coder.