The HTML 5 <video> tag and H.264

The new HTML5 standard’s most prominently mentioned new feature is undoubtedly the video tag – this tag enables all compliant browsers to play video embedded in a site with no additional plugins

The only problem? The organizations responsible for choosing a video format are undecided. In one corner is H.264, used everywhere from Blu-Ray disc to military applications due to its tremendous efficiency, and in the other is Ogg Theora.

Ogg Theora is known to be less suitable for content delivery. It requires higher bandwidth to deliver quality similar to that of H.264, which will increase infrastructure costs noticeably if universally adopted.

So why Ogg?

A common argument is patent encumbrance. H.264, while an international standard, is a creation of the Motion Picture Experts Group (MPEG), an organization known for their propensity to charge royalty fees to makers of DVD playback hardware and software. If the MPEG folks were to start asking for royalties from the Mozilla organization, for example, the latter would find itself in quite a dire financial position. This fear is borne out by prior attempts of large patent-holders to begin profiting from a wide deployment of their intellectual property, such as the Compuserve GIF format.

However, these past encounters with patent law also illustrate the impracticality, and indeed futility of these money making attempts. Neither GIF nor any other patented format has succeeded in exacting fees from end users or distributors of the mechanisms used to display or output these formats. MP3, a standard of the MPEG group as old as the laserdisc, remains cheap to implement.

The Motion Picture Experts Group has shown itself to be a responsible steward of international standards, reducing the concerns and possible benefits surrounding the Ogg format or other open-source codecs like it. As soon as H.264 is universally adopted, people everywhere can begin taking advantage of video as ubiquitously as they can view web pages. No plugins, no addons, no special devices, no more concern around compatibility.

Will it play anywhere? Yes. That’s a big deal, and it’s why H.264 should be accepted by everyone.

Home movies with Linux: Not Ready for Prime-Time

UPDATE: I had originally titled this article “Home movies with Linux: Beginning the exploration (of things Linux can’t do)”, but have since changed the title to be a bit more fair. It is a given that Linux isn’t suited to certain tasks, but rather than show these things up as embarrassing shortcomings I’d much prefer to demonstrate these areas of weakness to people who strongly believe Linux should replace everyone’s Mac / PC tomorrow. It can’t, and shouldn’t. We should all use our heads to: a) find the best tool for the job, b) recognize and put work into fixing shortcomings in our own tools.

Most computers now ship with a way to create and save videos. Youtube’s prevalence is an indication of how readily people want to take a movie clip, digitize it, and post it for others to see.

Criteria:

  • Installation of the OS and the app have to be as simple as they are on commercial operating systems.
  • The app has to take video in any format, arrange clips and save the result. I won’t even ask for special effects.
  • Preferably export to DVD.

After some browsing of gnomefiles.org and the ‘Add/Remove Applications’ panel, it appears a program called ‘Pitivi’ is most suited to this task. I decided I’d check it out from a fresh install of Ubuntu Feisty, as that distribution packages and distributes it. The following are notes on the installation and use of both Ubuntu and Pitivi itself from a usability and quality assurance testing perspective.

The Installation

Ubuntu Feisty (7.04) install went well. Asked me my name, country, and the current time. Installation completed in approx. 20 minutes.

Nicely done. Up there with the major operating systems so far.

Updates and Installation of Apps

  1. After install, ran updates. During download (not installation phase) of updates, changed themes from Ubuntu Human to Clearlooks. Update manager crashed without giving an error.
  2. Opened Add/Remove Applications in order to add software. Selected ‘Totem movie player (xine backend) and ‘mplayer-plugin’, which is used to play video embedded in web pages. Was greeted with the following:

     

    ubuntu conflict
    Jigga what?

  3. Moved on to select Pitivi, the video editor. Was greeted with another error:

     

    Ubuntu package conflict
    Is this going to happen every time? I don’t think my mom could use this to make a home movie.

  4. Taking the suggestion of these error windows, I closed Add/Remove Applications and switched to Synaptic. Was greeted by the following high-quality user-interface:

     

    synaptic
    Looking great. Hey, I know, why don’t we collapse all the information panes so they’re virtually meaningless! Oh! Already done! Thanks guys.

  5. Expanded the synaptic window and clicked search. Searched for and checkmarked the three items I wanted and clicked apply. Installation proceeded without issue. Closed synaptic.
  6. Used Places > Connect to Server… option to connect to another computer to upload screenshots. This worked once, but after disconnecting from this share I was unable to reconnect to it. I received no error. Even as an advanced user I was unable to resolve this and had to resort to SCP from Gnome Terminal. Eventually the desktop froze requiring a user to restart the computer as there is no way of logging out in this situation (menus were unresponsive) and shouldn’t be expected to know about ctrl-alt-backspace.(The correct fix for this is: ‘killall nautilus’ which I used, and again, an end-user wouldn’t know).

     

    Jammed
    Nice.

  7. Pitivi Itself

  8. Pitivi was unable to understand MPEG-4/H.264 files. Dragging these and other types of files onto the Pitivi sources list resulted in no error, but no import either. Documentation indicated that it should work. Ubuntu did not install Gstreamer components necessary to facilitate this. I installed these myself using synaptic.
  9. Synaptic remembered the window was maximized last time it was launched. Gold star.
  10. Installed: gstreamer-plugins-ffmpeg, gstreamer-plugins-bad, gstreamer-plugins-ugly. I only knew to do this due to prior experience. No obvious cues pointed out that these packages were needed. An end-user would, at this point, be completely lost.Import now worked. Audio playback, not so much. As it turns out I needed the multiverse variant of gstreamer-plugins-bad and gstreamer-plugins-ugly. I wouldn’t have known this if I hadn’t googled.
  11. During test playback of a 320×240 H.264/AAC movie, Totem did not properly antialias the expanded video. I don’t care which subcomponent is at fault, it looks like shit. Also note the green artifacts on the left. That isn’t present in the video stream.

     

    Totem antialiasing
    Totem botching up playback.

     

    quicktime-antialias
    QuickTime’s rendering of the same frame. Perceptually superior, no blockiness during playback.

  12. Pitivi continued to be unable to import .mp4 files by dropping the files on the ‘dragging them here’ area. I had to click ‘Add’ before they would import.
  13. Pitivi did not present any obvious way to re-export the imported video. It presented limited format options, restricting me to AVI, OGG, or MPEG container formats and AC3 or MP2 audio+MPEG-4 or Theora video. The ‘New’ ‘Open’ ‘Save’ and ‘Save as’ buttons and menu items never became active. Browsing the Pitivi wiki did not reveal a solution (or even cover the act of saving a project).

Pitivi documentation and setup is impractical for real-world use. While the user interface of the app itself represents a good start, it’s little more than a drawing on a napkin. While shipped with Ubuntu as an “easily installable” solution for video editing, the documentation for Pitivi clearly states that it can’t do the following:

  • Sources
    • Moving
    • Removing
    • Resizing
    • Cutting
  • Effects
    • Transitions
    • Video Effects
    • Audio Effects
    • Application-level plugins

Summary

Further research required on Linux-based positional video track editing/muxing. There appears to be no solution for this need as yet.

As for Ubuntu itself, little thought is being put into shipping applications in a way the user can easily install. All visual cues point to Add/Remove Applications, which doesn’t work. Synaptic’s user interface is abominable. Dependencies are not properly installed, resulting in the need for expert intervention. The desktop environment can’t reliably perform tasks such as copying files to another computer. On top of this, Ubuntu is shipping apps that aren’t even fully functional yet. What good is a video editor that can’t move/remove/resize/cut video?

No Linux reality-check article is complete without a humiliating comparison with software designed by paid professionals. For an example of an application that _comes with_ a commercial operating system and allows you to make movies with sophisticated effects and import/export options, check out iMovie – Making Movies/Creating a movie.

Just because you’re unhappy with Windows doesn’t mean you have to go running off to something even less functional. Remember, choosing an alternative to Windows isn’t about hype, it’s about making smart choices.

HOWTO: Rip DVD’s in MacOS X (Updated)

This HOWTO has been updated and dramatically streamlined! It’s worth re-reviewing if you’ve read/implemented this method already. Keep in mind, this is for experts/enthusiasts and not those looking for a quick one-click solution.

The quality of video and sound is, for the average viewer, highly subjective. When it comes to viewing a DVD though, one can be certain that they’re watching the highest quality (until HD-DVD/Blu-Ray standardize) video they can get.

Another given is that when making archival copies of a DVD (DVD9 -> DVD9) there is no degradation in quality. Increasingly though, personal computers are taking the role of a media center hub, and one way they can effectively do this is to store a catalog of one’s movie collection for easy access (and no need to have a multi-disc-changer).

For the purposes of this we’re going to assume that while ripping a DVD to DiVX can be accomplished fairly easily with HandBrake in three steps, you want more ability to tweak and fine-tune the end-result to look far better.

Here I’ll provide an overview of the way I’ve found most effective in converting a DVD to a movie file with the highest possible quality. This method also has the benefit of using Apple’s QuickTime software for every stage of video conversion. QuickTime technology is used in the production of almost every movie Hollywood makes, so why use anything less to convert them?

I’m going to assume you have a basic familiarity with how to read manuals and familiarize yourself with the operation of a program too. 😉

The Tools:

The Steps:

I’m assuming here that you have a DVD with unencrypted content or the contents of a decrypted version already handy. If you’re within your usage rights to decrypt a movie (Fair Use laws apply!) you can use Mac the Ripper to do this.

Next we use Cinematize to convert the feature from VOB to MPEG-4. We’ll be using its Lossless setting to do this, so the conversion will lose as little detail as possible (there will be some, but it will be so slight as to only be measurable by professional equipment).

Cinematize Settings:

Drag the VIDEO_TS folder into the drop box in the main window. You’ll be presented with a list of items, one of which will be ‘Video Title Set 01’. Select it and click the ‘Select VTS’ button.
Segment tab: Title 1, Angle 1; Start Point: Chapter 1, End Point: Whatever the last chapter is.

Video Tab

  • Video Stream: Main Video Stream
  • Decoding menu: Decode To QuickTime
  • Decoding Mode: Automatic Selection
  • Output Codec: MPEG-4 Video
  • Output Quality: Lossless Quality
  • Aspect Ratio: No Size Adjustment

Audio Tab

  • Audio Stream menu: Audio 1
  • Decoding menu: Elementary Stream

Output Tab

  • Save Chapters: Together as One Segment
  • Output Format: Separate Stream Files

On a relatively new Mac it will take approximately 2 hours to perform this step (slightly faster than realtime). The output will consist of a .mov file containing the video, and a .ac3 file containing the Dolby audio.

Open the .mov file in MPEG Streamclip. Once you’re sure it’s in good shape, go to the File menu and choose ‘Export to MPEG-4…’.

MPEG Streamclip Settings:

  • Compression menu: H.264
  • Check the following boxes:
    • Multipass
    • B-Frames
    • Limit Data Rate
    • Pick a value for the data rate. Higher bitrate means higher quality, but also more disk space. Remember that H.264 is 3-7 times more efficient than MPEG-2, which was the source format. The DVD bitrate is 7500, so you should never set the date rate this high or higher. You can safely choose between 1100 and 2500 for bitrate without compromising on perceptual quality.

  • Quality Slider: 100%
  • Frame Size: Unscaled
  • Place a check in the ‘Cropping’ box.

In the previous revision of this HOWTO I gave you the choice of H.264 or DiVX. Now I’m not. H.264 is clearly superior. Everything from the iPod to the Xbox360 has adopted it for high-def playback, so it’s time to let DiVX go.

Removing Letterboxing

This step requires a bit of experimentation. In order to trim off the black bars above/below the picture you have to start by guessing at the values to place in the top/bottom fields in this dialog. You can preview the results by temporarily unchecking ‘multipass’ and pressing the ‘preview’ button on the bottom right. This will run through the movie until you press ‘stop’.

Cropping rules:

  • The values for top and bottom must add up to an even number
  • Preferably (not required) a multiple of 16

Repeat this until you don’t see black bars above and below. Don’t worry if you’ve removed a couple of pixels from the top/bottom. Overscan is a common practice in video, so movie makers don’t put anything important in the top row of pixels anyway. 😛

Remember to checkmark Multipass when done.

Click ‘Make Movie’. This will take a while. On a relatively new Mac about 7 to 9 hours will pass.

Once the movie is exported to H.264 format you’ll find a .mp4 file in the location you saved it to. Open this with QuickTime Player. Be sure you’ve registered for QuickTime Pro, since we’ll need these features. Also open the .ac3 file generated by Cinematize.

With the .ac3 file in focus, choose ‘Export’. In the Export file dialog box that opens, pick ‘Movie to QuickTime Movie’ from the drop-down and press the options button.

Movie Settings

There are two areas here, Video and Sound. Video will be grayed out, as obviously the .ac3 file contains no video track. This is fine.
Click Settings in the Sound area.

  • Format: AAC
  • Channels: 5.1 (C L R Ls Rs LFE)
  • Rate: 48.000 kHz
  • Check ‘Show Advanced Settings’
  • Render Settings – Quality: Best
  • AAC Encoder Settings – Bitrate Format: Constant Bit Rate
  • Target Bit Rate: 80 (this is per-channel)
  • Precedence: Bit Rate.

Save this. Approximately 30 minutes will pass. Open the resulting .mov file.

Choose ‘Select All’ from the Edit menu and then ‘Copy’. Switch focus to the .mov file containing the video and make sure the play position is right at the beginning. Choose ‘Add to Movie’ from the Edit menu.

Note: In some cases the audio can be slightly longer than the video, resulting in white frames at the end of the movie. You can use QuickTime’s graphical controls to trim this off (it’s usually only a few milliseconds worth anyway).

Play through a few sections of the movie to make sure audio and video are lined up. If so, Save As… and make a self-contained movie out of the result.

Anamorphic PAR

You’ve probably noticed by now that everything is too tall. This is because DVD’s typically store video in anamorphic form, meaning the pixels making up the video do not have a set shape. They’re stored vertically stretched on the DVD, and are to be stretched horizontally to fit the viewer’s preference for widescreen playback. This is known as ‘Anamorphic Pixel Aspect Ratio’ or Anamorphic PAR.

In this case we want to preserve the widescreen aspect of the movie. Fortunately this doesn’t require guesswork. The video you’re currently working with is 720 pixels wide. For NTSC movies the widescreen mode is 854 pixels wide, and for PAL movies the width is 1024. Height is variable and depends on the black bars we trimmed off.

To adjust this, show the ‘Movie Properties’ window from the Window menu, select the video track in the list, and click the ‘Visual Settings’ tab. Uncheck ‘Preserve Aspect Ratio’ and set the value of the first field to either 854 (if the source came from North America) or 1024 (if it didn’t). Close this window and save the changes.

Optional: Compatibility with other players

This is strongly recommended, as it will enable other players to display the correct aspect ratio. Open the newly created movie with ‘Dumpster’. Be careful with this app, as one wrong movie can corrupt the movie file and you’ll have to reassemble it in QuickTime from the source files.

Dumpster lists the internal structure of the .mov file. You’ll find:

‘moov’ – Movie
‘trak’ – Track

Expand the ‘trak’ item by clicking it. This will reveal a ‘tkhd’ subitem, expand it as well by clicking it.

Dumpster

Within the fields here you’ll find ‘trackWidth’, ‘trackHeight’ and ‘matrix’. Set the ‘trackWidth’ to the values prescribed above, and reset the ‘matrix’ value to 1.0 (the ‘matrix’ is what QuickTime uses to adjust the aspect ratio, but other players don’t understand it. Fortunately, QuickTime understands what we’re doing here). Carefully noting that nothing else has changed, save and quit Dumpster.

That’s it, we’re done. To recap:

We’ve extracted the MPEG-2 video and Dolby audio from a DVD and converted it to H.264 using QuickTime (via some applications that use QuickTime to do their heavy lifting while providing value-add features), preserving the surround sound and converting it from the older Dolby AC3 to the newer MPEG-4 Advanced Audio Coding method. Enjoy your new movie.

Proper H.264 Encoding

Hopefully people will glom onto this information, as it’s sorely lacking. Here’s the upshot – DiVX and XViD are dead. H.264, more commonly known as MPEG-4/AVC is the new champion of compressed video. And yet, most media files are still released using DiVX and XViD, chiefly because people are not totally familiar with how to create H.264 files.

What’s worse, many people simply use graphical frontends to encoding engines, which don’t provide them the newest options for H.264 encoding.

The primary option for encoding movies into H.264 MP4 is QuickTime and MPEG StreamClip. These two are available for both MacOS X and Windows, and provide feature parity for MPEG-4 generation with this codec. You can get QuickTime from Apple’s website, and MPEG StreamClip from Squared5.

MPEG Streamclip Mac

MPEG StreamClip’s options window for exporting to H.264.

MPEG Streamclip Windows

Windows version of the same app – options are identical. Image credit: Chad Lewandowski

Another excellent option for encoding this format is MEncoder, part of the MPlayer application. Support for H.264 has recently stabilized to the point that it is able to generate H.264 / MPEG-4 content capable of being played back in other compliant players. It does have the drawback of being unable to presently encode using B-Frames, but this will hopefully be corrected soon. This tool is a command line based application, but the options are easy to comprehend and are specified in its documentation. Typically you want to enable most of them. As such, the following command line will generate high-quality output in most cases:

mencoder -of lavf -lavfopts format=mp4:i_certify_that_my_video_stream_does_not_use_b_frames -oac lavc -ovc lavc -lavcopts aglobal=1:vglobal=1:vcodec=h264:acodec=aac:vbitrate=800:abitrate=128:v4mv:coder=1:qpel:trell=lambda:mbd=1:vme=1 -vf harddup

Where vbitrate and abitrate control the video and audio stream bitrates respectively. vpass= can be passed to the encoder as well. On UNIX based workstations with a properly installed MPlayer package, these options can be further explored.

Hopefully this is of assistance to people looking to create media files. When looking for a video codec to use, don’t look any further than H.264. It’s the state of the art.

H.264, how do I love thee?

H.264, MPEG-4’s advanced codec featured in the iPod, the Sony PSP, and both Blu-Ray and HD-DVD, as well as future broadcast video standards, has a lot of skeptics. Many say H.264 doesn’t have a lot to offer people when videos are compressed at the higher bitrates required for HD content (self-contradictory, as H.264 is accepted as the standard MPEG-4 codec for High Definition).

To them I just want to say:

This is your brain,

quality.png
MPEG-4/AVC (H.264, 700kbps) – click for larger image

And this is your brain on drugs:

quality2.png
DiVX 6.1.1 (DiVX MPEG-4, 700kbps) – click for larger image

Any questions?