From icculus@lokigames.com Thu Jan 25 04:15:09 2001 Date: Wed, 24 Jan 2001 05:18:11 -0800 (PST) From: Ryan C. Gordon Subject: An mpeg success story. I'm now about a week deader, but I manage to convert the Descent 3 intro movie into a file that SMPEG can grok. Here's what I did, from start to finish. Perhaps this will help y'all at sometime in the future. Who knows? First, I had to hack the source to Descent3. In the movie playback code, everytime I render a frame, after SDL_UpdateRect() gets it to the screen (this will be your progress indicator), I call SDL_SaveBMP() to dump it to disk, and increment a counter used in the filename. I also had to disable the audio sync'ing code, since I wouldn't want this to skip frames due to disk writing lag. After this work, I just had to sit back and watch the movies in slow motion. After the intro, I ended up with a directory containing 7780 bitmaps named frame0.bmp, frame1.bmp, etc. Now, I needed to rip the audio track. There is a program that comes with esound called esdmon. It gets information from the esound daemon, and dumps all sound data that is to be played to the sound card to stdout. So I started up esd, and exported SDL_AUDIODRIVER=esd to make Descent3 play it's soundtrack through esound. I also flipped the condition in the Makefile that compiles in the dump-movie-to-disk code, since that seemed to turn off the movie sound playback entirely. Whoops. :) esd & esdmon -s localhost > audio.raw export SDL_AUDIODRIVER=esd ./descent3 --options... ...then, watch the movie, and kill the process right after the movie finishes to prevent more sound from going to esd. Descent doesn't play anything more until everything is loaded, so there's plenty of time to CTRL-C in this case. Otherwise, you can try to lop the end of the audio track off in Broadcast2000, later, or hack the source to your game. I had to chop off the first line of the raw output in a text editor that handles binary files, since esdmon writes an intro banner to stdout. After this, I used "sox" to convert from raw data to WAV format. sox -t .raw -r 44100 -s -w -c 2 audio.raw -t .wav audio.wav (The "-t .wav audio.wav" can be replaced with "-t ossdsp /dev/dsp" if you want to test your command line options by dumping to the soundcard.) Update: I've written an SDL audio driver that writes directly to disk, so you can dispense with this esd nonsense. Other things not using SDL (but can use esd or the esddsp wrapper) might still get some mileage out of the above, so I'll leave the info there. With SDL apps: Build an SDL library with --enable-diskaudio passed to the configure script. export SDL_AUDIODRIVER=disk export SDL_DISKAUDIOFILE=/where/i/want/the/output/sound.raw Run your SDL program, and watch your disk fill up. Also, if you want it to write out as fast as possible (like the XMMS diskwriter plugin does), export SDL_DISKAUDIODELAY=0. If you want it to write at something more normal (for trying to sync to a video, or to get the correct effect from the panning test code in SDL_mixer's playwave), try higher values. It is specified in milliseconds it should pretend to be waiting for the "sound card" to finish writing its buffer, and defaults to 150 milliseconds. Data is written to disk in whatever format the SDL application asked for when calling SDL_OpenAudio(). You can use sox to convert the raw data to another format, or play it to the soundcard as such: # Assume the raw data is 44100Hz, 2 channels, 16-bit signed data. sox -t .raw -c 2 -r 44100 -s -w sdlaudio.raw -t .wav output.wav sox -t .raw -c 2 -r 44100 -s -w sdlaudio.raw -t .ossdsp /dev/dsp Nothing I could find (including Premiere 6.0 on win32, which segfaulted about halfway through loading all this, and Scott's MPEG encoder on MacOS, which couldn't even open the directory with 7000+ files in it over an AppleShare. :) ) could do anything with the graphics in BMP format. There is a Linux program called Broadcast 2000 (http://www.heroinewarrior.com/), which is a bitch to use, but had sorta the functionality I needed. This could make a movie out of JPEGs. Broadcast2000 needs a text file describing the sequence of jpegs to use. Not very user friendly, here. The text file needed to have the word JPEGLIST as the first line (that's the "magic number" for the bcast2000 plugin), the number of frames per second on the next ("30.0", in this case), width after that, and height after that. Every line following was the next file in the animation sequence. So I needed to make this file, convert all those bitmaps to jpegs, and halve the size of the video (after all, the gameplan is a loss in visual goodness for faster playback and smaller datasize). This was best handled by your friend and mine, Perl. Code follows: ----- #!/usr/bin/perl -w use strict; my $i = 0; my $curdir = `pwd`; chomp($curdir); open(JPEGLIST, ">jpeglist.txt") || die(feh $!); print JPEGLIST "JPEGLIST\n"; print JPEGLIST "30.0\n"; print JPEGLIST "320\n"; print JPEGLIST "240\n"; for ($i = 0 ; $i < 7710; $i++) { print "Frame #$i..."; `convert -quality 100 -scale 320x240 frame$i.bmp x$i.jpg`; unlink("frame$i.bmp"); print JPEGLIST "$curdir/frame$i.jpg\n"; print "done.\n"; } close(JPEGLIST); ----- "convert" is part of the ImageMagick package. Note that there's a "-quality" command line on there to prevent loss in graphic quality. We'll save that for later, so that the compression loss is consistent looking across all frames. Regardless, several gigs of bitmaps was reduced to much, much less by this process. Fired up bcast2000.sh, and loaded the jpeglist.txt file. The first couple of frames (complete blackness) rendered with visual artifacts unless I explicitly "muted" the video track on that section (it's under the "edit" menu)...I guess this is a bug, but it was a small problem. Select "render" from file, and render it to an uncompressed Quicktime format. I set the quality to 75%, partially because Quicktime movies have a signed 32-bit limit in size, and full quality on the intro put it just over that 2 gig limit. Download MPEG2MOVIE from heroinewarrior.com. Build it. Run thier wav->mp3 converter (it's just the LAME program). Be sure to encode at 44.1KHz, or the multiplexing won't work later. Feel free to experiment with bitrates, but I found that 128kb didn't bloat the final MPEG too much, and sounded basically perfect. /where/i/put/mpeg2_movie/audiomp3/lame -h -b 128 --resample 44.1 [in] [out] Also convert that big-ass uncompressed Quicktime to MPEG-1 video. 1.6 gigs became about 15 megabytes. Tinker with the settings, but I found this worked well: /where/i/put/mpeg2_movie/video/encode -p -1 -q 10 -n 300 -m 0 [in] [out] The -p is "progressive encoding", -1 tells it to encode an MPEG-1 (instead of an MPEG-2) video, -n and -m control space between I and P frames, whatever those are. The -n setting reduced the size of the file a little without any noticable change in quality. The only real tinkering point is the -q argument. No argument will give you a MPEG that's huge (but it looks REALLY good), 10 gives you about standard MPEG quality, 20 makes it really blocky, and > 20 will basically give you colored shapes for video. 20 gave me a final video size of about 8 megs. 10 gave me about 15 megs. No -q option gave me a prediction of about 100 (but I cancelled it, so who knows?). I evetually settled on somewhere between 5 and 8 for most movies. After you've got an MPEG-1 video stream, and an MP3 audio stream, you multiplex them. /where/i/put/mpeg2_movie/mplexhi/mplex video.mpg audio.mp3 finaloutput.mpg Multiplexing, unlike the rest of this process, only takes a few moments. The mplex program in the "mplexlo" directory claimed success, but I didn't get an audio track in plaympeg. mplexhi will choke if you give it an mp3 that isn't encoded in 44.1KHz. That's all. The only tragedy here is I will never get a Nobel prize for what it took to accomplish this. :) --ryan.