I found the Android application gmote on the Android Market and thought I'll check it out. I have a big screen TV that is used as a display for the computer. Like most I run VLC Movie Player.
The installation of the host is pretty straight forward. Works all OS too, get the Gmote > Server deamon.
Once you run the file, you will be asked which directory the client should connect to. Afterwards the computer should operate as a host.
And indeed the phone application found the computer running Ubuntu. I could use the browse to navigate through my folders and it was pretty easy to start something. All the keys (start, stop, pause, volume, mute, forward and backward, as well as off) worked well and surprisingly responsive.
Use the browse to load a movie and off you go.
Interesting benefit: there is a beta to play mp3 and mp4 and I did try to play a video, but the video would not load on my phone. Maybe the mp4 was way too big and complex to buffer? I don't know...
Maybe I will try a smaller one some time.