Quantcast

Star Wars re-remastered to high quality 1977 version!

So I wanted to sit down in my updated home theatre and watch one of my favorite films, “Star Wars”. Problem is, I have two versions on DVD, and neither are ideal. The 2004 DVD version has remastered audio and video, but also added scenes and changes I really don’t like. The 1977 DVD version is a poor quality transfer and encoding.

So I combined the 1977 and 2004 DVDs into the highest quality 1977 version of the film! I show all my scripts and techniques here, too, so you can make the same version from your two DVDs, too!
http://www.archive.org/details/reremaster

star wars comparison of two films

star wars comparison of two films

ffmpeg building on mac intel/x386

ffmpeg v0.5 just came out. it’s the bomb. it’s got tons of fixes and massive amounts of new codecs that it can read. for example, it can now decode my professional filmmaker brother’s “DVC ProHD” highly proprietary (and massive bitrate!) format! it can also decode flac and 24-bit flac. (encoding flac is disappointing though).

at any rate! macports is a great way to get it installed on your mac.
the current way of setting up macports and then doing
sudo port install ffmpeg
works fine on my PPC at work (oddly — pretty old computer now) but not my Air (intel x386)

So I set out to find the fixes needed to make it work. Here they are:

step 1: install macports — see http://www.macports.org/install.php

sudo port install x264 +noasm # for i386 (not needed for PPC)
sudo port fetch ffmpeg
sudo port checksum ffmpeg
sudo port extract ffmpeg
sudo port patch ffmpeg
remove “–enable-shared” from /opt/local/var/macports/sources/rsync.macports.org/release/ports/multimedia/ffmpeg/Portfile
sudo port install ffmpeg

(and “ffplay” is pretty cool now! i am considering using it over mplayer, hmm….)

ffmpeg hook to aid with “rectangular pixels”

Tired of screen-scraping the output of ffmpeg and/or mplayer to get the parameters / clip info for a media file?

This hook attempts to remedy this by printing simple information about the passed in video from the cmd line.

It will also print out whether or not the clip is using “rectangular pixels”.

WTF is a rectangular pixel?

 

Well, the easiest examples are DVDs.  You only want to buy a DVD if it says one of two standard phrases on it — “Enhanced for 16:9 TVs” or “Anamorphic widescreen“.  They both mean the same thing — namely that the video on the DVD disk is wider than 4:3 aspect (pretty much all films are ratio 16:9 or even wider like ratio 2.35:1) *and* that it didn’t *waste* any DVD bytes by encoding “top/bottom black bars”.  (If it doesn’t say those code phrases, it’s an older, crappy/low budget produced, or worse a “pan-n-scan” chopped film (where they lop off the left and right sides of each frame to fit into a 4:3 TV!)  Worst yet, if the DVD only says something like “widescreen version”, though it sounds good, it means while they didn’t cut off the picture, they wasted 25% (or more!) of the pixels encoding “black bars” on the top and bottom.  So you have less pixels in the DVD encoding the picture compared to an “anamorphic” version of the same thing.  Hello crappy quality!)

 

 

Anamorphic DVDs are encoded internally at 720×480 pixels per image.

Now look at this:

  4:3 video == 1.33 ratio == 640×480 pixels image

16:9 video  == 1.78 ratio == 854×480 pixels image

so what is 720×480?  it’s almost perfectly in the middle of those two — math: 720/480 = 1.5

So the “encoded transport image size” is neither 16:9, nor 4:3 — it’s right in the middle, capable to encode either a 4:3 video (like non-high def TV, many computer screens) or 16:9 video (high def TV, some digital video).  I like to think of it as “how the formats that came out right before HDTV took off, compromised to hedge there bets to be between 4:3 and 16:9″.

The final critical bit of information about a DVD is the “aspect ratio” (not of the overall image, confusingly, but of each encoded *pixel*!)  This says “wait, this pixel isn’t square, literally, like you’d think if you just read the image — it’s actually supposed to be stretched to make the *overall image* either 640 pixels wide (squoosh down from 720) or 854 (stretch wider to 854).  So the video track is “flagged” with a “pixel aspect” (often referred to as PAR (Pixel Aspect Ratio), SAR (Sample Aspect Ratio), or DAR (Display Aspect Ratio) — some of those have some slight nuances/differences, but that’s digging too deep).  Anyway, neat, huh?  Your anamorphic DVD is a changeling!  (maybe “anamorphic” makes more sense mnemonically now 8-)  (PS: “DV” video — the most common format that digital camcorders that write to tape use — is similar.  720×480 pixels/image plus a “flag” for what each pixel “shape” is).

 

This hook I wrote will output information about the clip (like “ffmpeg -i” will do, but in a format easier to parse) as well as information about the pixels (unlike ffmpeg).  So you can then know more about a clip if you are going to do things like pull single frames/thumbnails from it or convert it to another format.  We have been using this at Internet Archive for our movies for over a year and a half now and it works great!

 

C Code is here.  

There are some instructions for compiling with an ffmpeg source at the top of the C code.  (There is also an ubuntu-on-AMD compiled “.so” at that link by changing the suffix from “.c” to “.so”, FWIW)

 

Example invocation and output:

ffmpeg -vhook "/petabox/deriver/identify.so oldpresidio.mpeg"
FFmpeg version SVN-rUNKNOWN, Copyright (c) 2000-2007 Fabrice Bellard, et al.
  configuration: --enable-gpl --enable-pp --enable-libvorbis --enable-libogg --enable-liba52 --enable-libdts --enable-dc1394 --enable-libgsm --disable-debug --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-x264 --prefix=/usr/
  libavutil version: 49.3.0
  libavcodec version: 51.38.0
  libavformat version: 51.10.0
  built on Nov 30 2007 19:09:20, gcc: 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)
Video: mpeg2video, yuv420p, 720x480, q=2-31, 8000 kb/s
width: 720
height: 480
aspect: 32/27
fps: 29.97
duration: 00:03:05.2
audio: true
Failed to Configure /petabox/deriver/identify.so
Failed to add video hook function: /petabox/deriver/identify.so oldpresidio.mpeg
The “failure” above at the end is deliberate/OK (it just makes sure ffmpeg stops and doesn’t try to transcode).
So we see that this video clip:
is indeed widescreen by the “aspect” line above (indicating the pixels are rectangular, not square) with value 32/27.
If we multiply the encoded image width of “720″ by 32 and divide by 27, we get the magic/correct 853.33 (round up or down to nearest pixel).
We use this utility at Internet Archive to make user friendly formats like “h.264 .mp4″ videos and “Ogg Theora .ogv” videos that get converted to the proper square pixel equivalent (and *not* messup widescreen videos 8-)

Timelapse: OSS Secrets

I thought I would share a couple of helpful GPL command lines that really get the job done if you want to animate a sequence of images.

First (via Mike) is news of the jhead project. Unbelievably the ImageCapture.app application that ships with Mac OS X cannot handle more than several thousand images. ImageCapture.app will copy all of the image files, but the number sequence wraps around and it overwrites thousands of the earlier images in the sequence!! jhead fixes all that by renaming all of the image files based on the EXIF timestamp in the JPG file. brilliant! Here is the commandline he recommends:

jhead -nfpx%Y%m%d-%H%M%S -dt -ft $*

Now you have a nifty sequence of hopefully hi-res images. One of the cool things about timelapse is you can shoot 5 megapixel images and then scale them down to NTSC (720×480) or HD (1920×1024) resolution. In fact if some crazy 5 megapixel video format comes out 10 years from now, you will be able to support that as well! (As an aside, the Digital Cinema releases that are dribbling out to theaters these days are at 2,000 pixels wide. Eventually that will go up to 4,000 pixels wide, but there are not currently projectors that can support that resolution!)

So anyway, megapixels. How do you scale the images without fudging it up? A little known fact is that most video that looks crappy because of “the video compression” actually looks crappy because of the way the video was scaled down to its postage stamp resolution. So here is what you do: Use the fantastic ImageMagick “convert” command to scale your images down at even higher quality than photoshop! for free! and without using a dumb GUI scripting language. You can read about image scaling in laborious detail or you can just use this command line:

convert  -filter sinc -resize 720x480 foo.JPG foo_ntsc.JPG

I put the following bash script into a makefile to process my timelapse library:

for i in *; do  if [  ! -e "$i/ntsc_jpg/" ]; then  mkdir "$i/ntsc_jpg"; for i in orig_jpg/*.JPG; do echo $i; echo convert  -filter sinc -resize 720x480 ntsc_jpg/`basename $i`;  done fi done

And finally I would like to share the “ghetto HD” format that VJ Science came up with for doing 16:9 format on an XGA projector. This command includes cropping to get my 1600×1200 images into 16:9 aspect ratio … if you have a different source resolution, I’m going to leave it to you multiply your image width by 9 and divide by 16 to figure out the how many pixels to crop on top and bottom. The idea is to crop first and then scale down. Although I guess in terms of quality it doesn’t matter the order.

convert  -crop 1600x900+0+150 -filter sinc -resize 1024x576 foo.JPG foo_xga.JPG

Now you’ve got a nifty sequence of beautifully scaled images. Time to make video! Which of the 7 bajillion video codecs are you going to use? My recommendation is to use ffmpeg to make an mpeg-2 “master” that you can distribute by uploading to youtube, archive.org, and so on. ffmpeg understands image sequences! So here is a quick way to get your mpeg-2 video (no audio). This command assumes your images are named like “IMG_0001.JPG” and so on … I know that doesn’t jive with the jhead naming I’ve outlined above, but you are a big unix haxor and can figure that out.

ffmpeg  -i "IMG_%04d.JPG" -b 10000 test.m2v

For my purpose I am building up a video library for doing VJ/video mixing stuff. In this use case video quality and decode speed is much more important than bitrate. Since decoding mpeg-2 “I” frames is about 3x faster than decoding “P” or “B” (difference) frames I use this command to get a high quality I-frame only mpeg-2 stream:

ffmpeg  -i "IMG_%04d.JPG" -vcodec mpeg2video -intra -qscale 4 test.m2v

You can get higher quality by dropping qscale down even lower, but I think if you do qscale=1 your video basically won’t be compressed at all :-) rajbot?

So there you have it! The basics of an Open Source timelapse production pipeline. I use these tools on Mac OS X. Of course they are available on GNU/Linux. And you can also get them working under windows using Cygwin … We need more timelapse!

Link to jhead project
Link to ImageMagick project
Link to FFmpeg project