PS3 MP4 Encoding with Linux

After a lot of reading, searching, and trial and error, I have come up with a fairly simple way to convert video and audio content into an h264 and aac stream. The stream is packed into an MP4 file and can be streamed to a PS3 via a media server such as MediaTomb.

The following scripts makes it a pretty automated process. It relies on mencoder, and MP4Box. I have been using it to convert DVD vob files into smaller MP4 files, with nearly the same quality. I can't actually tell a difference from the original DVD source.

#!/bin/bash
VIDEOFILTER=crop=704:480:8:0
ENCOPTS=subq=5:bframes=4:b_pyramid:weight_b:psnr:frameref=3:bitrate=$2:turbo=1:me=hex:partitions=all:8x8dct:qcomp=0.7:threads=auto

mencoder -v \
$1 \
-alang en \
-vf $VIDEOFILTER \
-ovc x264 -x264encopts $ENCOPTS:pass=1:turbo=1 \
-ofps 24000/1001 \
-vobsubout "$3_subtitles" -vobsuboutindex 0 -slang en \
-passlogfile "$3.log" \
-oac copy \
-o /dev/null

mencoder -v \
$1 \
-alang en \
-vf $VIDEOFILTER \
-ovc x264 -x264encopts $ENCOPTS:pass=2 \
-oac faac -faacopts object=1:tns:quality=150 \
-passlogfile "$3.log" \
-ofps 24000/1001 \
-o "$3.avi"

MP4Box -aviraw video "$3.avi"

MP4Box -aviraw audio "$3.avi"

mv "$3_audio.raw" "$3_audio.aac"

rm "$3.mp4"

MP4Box -isma -hint -add "$3_video.h264#Video:fps=23.976" -add "$3_audio.aac" "$3.mp4"

rm "$3_video.h264" "$3_audio.aac"

Running the script involves something like...

./x264Encode2.sh Terminator2.vob 3800 Terminator2

The first argument is the source file, the second is the target bitrate, and the 3rd is the target file. I do not put an extension on the target as the script will do this. A bitrate of 3800 seems high enough to produce an encode that looks nearly identical to the source DVD.

You will probably need to tweak the VIDEOFILTER crop filter according to your video source. You can run mplayer on your source using -vf cropdetect to determine the correct cropping arguments. Of course you can also change the ENCOPTS to your liking, although I find the current options acceptable.

Once the oncode is finished you have a nice MP4 file. The avi file is left after the encode in-case something go wrong so you don't have to re-encode the source. If everything goes ok, you can delete this file.

Let me know what you think.

EDIT: The PS3 seems to be picky about the video resolutions, be careful when cropping to strange resolutions. I'll try to investigate this more, as I ran into a problem with a cropped wide screen video. Removing or tweaking the crop made a short test encode work.

Callbacks, the trick pointer you execute

At work I tend to develop programs using C#, which I don't particularly like but it does have some neat features. One of the features that caught my attention are delegates. Delegates are essentially function pointers (actually they do a little more than that). A function pointer is just a pointer to a function much like a pointer to a variable. I found it extremely interesting how delegates are used in ASP .NET for event driven processes.

I began wondering if I could implement similar systems in my own projects using C++.

Callbacks in C++ provide a way to call functions by executing function pointers. In C# delegates are type safe, this helps ensure you don't end up passing and receiving junk data to and from your function. Callbacks are not quite as nice, they are not necessarily typed and usually make heavy use of void pointers and unsafe type casting. To achieve typed callbacks in C++ I started making a templated callback class that allows for typed arguments and typed returns. Using a templated library helps ensure type safety and provides excellent code reuse when using callbacks with various types. Using the templated callback library, making a callback is as easy as...

void myFunction(int x)
{
...
}

Callback1<int> myCallback( myFunction );

Executing the callback is then as simple as

myCallback(7);

Why would we want to go through all this trouble just to run myFunction? Well, think about a system that executes functions but the functions are not known until runtime. Event processing tends to be one such system.

For event processing you could have a function that contains one giant if else or case statement for every possible event. Or you could make a system where you register functions with a type of event, and when the specific event occurs, a callback to your function is executed. This type of event system is exactly what I used callbacks for.

In the Stepmania Online game server I am developing, players are able to enter commands from the chat interface to manipulate the server. Since the events the server is handling are text strings, a switch statement is not an option. With a couple commands, multiple if else statements is an ok solution. However once the number of commands begins to grow, the if else statements become very long and ugly. To solve this I made a list of callbacks, each associated with a string command. I then "registered" each function with a string command by creating a callback to the function and adding it to the list along with the associated string command.

When a command is received, the program simply loops through the list of command stringsand callback. When a match is found between the stored string and the command, the callback associated with the command is executed. No giant if else statement required!

The real power in this system is that extra commands can be created by just compiling and linking in extra commands. This means, the game server could be released as a library and the chat commands can be expand uppon without touching or recompiling the core of the server. Very useful!

I have found that it takes a while to fully grasp the concept of callbacks, but it is well worth the effort.

Revisiting old code

This past week I began working on Mercury2, which will be almost complete rewrite of my first game engine Mercury. So far I have only worked on the render window. While working on it i decided to look at my old code from Mercury1 to try to figure out how I had done it. When looking back at old code, its amazing to see how much your coding style changes and improves over the years. Its been over three years since I wrote the windowing code for Mercury and it certainly shows.

Looking at my old window code, it was clear I didn't have any understanding of what I was doing when I wrote it. Actually a lot of it was taken (and giving credit to) from NeHe lesson 1 and was modified to be a little more flexible (in my mind at the time). The result worked well, but the code was a pretty convoluted. I decided it was best to sit down and review my old code to try to figure out exactly how it was supposed to work. So with a long of review of my old windowing code, and some heavy reading of MSDN documentation I developed a much more solid window system.

Moving forward in the project I think it will be difficult to properly design the project knowing I have large amounts of "old code that works" that I could fall back on.

Open source development vs. business development

Over the past eight months at my current job I have had an opportunity to work with various people in designing new programs to fit their business needs. While working on the projects, one aspect has become clear, developing free open source software is vastly different from developing software for your employer.

The writing of the software isn't so different. You still need to plan your software(based on requirements), write it, and then test it. In my experience, many open source projects are planned rather poorly. They are then initially coded up rather quickly to a point of working "well enough". The programs tend to be somewhat tested by the developers but rely heavily on "community testing." This often results in more than a few negative user experiences. Development of many open source programs begin to stagnate after a while and end up in a continuous state of development, never really being done.

In business environments, software development I have found works differently. Planning is done with a "client". Rarely does the client actually know the full requirements of the software they are requesting. I generally try to understand the client's current business process they are trying to enhance/automate etc. While discussing the business process with the client, we try to determine the actual requirements based on the business process. (I have found that the client doesn't always know the complete process or may forget parts. Its good to involve a few people that know and work with the process.) One fact about the software requirements, they will change throughout a project, no matter what.

After the requirements are known, the real work can begin. However unlike most open source software there is a very real and probably very near (closer than you would like) deadline. There is little time to be spent refactoring poor inflexible code. Its very useful to take the time upfront to really think about your approach, you also need to plan for testing. I have found that testing generally takes just about as long as writing the program. This is not necessarily because of a large number of bugs but rather who is testing and how much attention they give to it. I like to involve the client in the testing process as they know the process the best. They are the ones who will be able to spot logic errors in the process. The clients tend to think about the process much more differently than myself as the programmer, what was clear to them may not have been clear to me. Anyhow, testing... allow ample time for it. I often find myself and the client testing right up to the deadline. Even after the deadline problems are often found while the software is being used in a production environment. Testing is very important when your software will have a real impact on businesses and user impressions of that business.

A Little Background

Currently I'm a software developer working for a small college. I work with a variety of systems including HP Unix , Mac OS X, and Windows. On these various systems, I program using perl, php, and C# .NET while developing web based applications.
In addition to my current position I have worked extensively on several open source application including Stepmania, Stepmania Online, and The Mercury Game Engine. These projects have incorporated many different programming techniques and paradigms. I'm am always experimenting with new (to me) programming techniques.
This blog will be focusing on interesting aspects and discoveries made during the course of my software development.