Running Systemd within a Docker image

NOTE: This is not for general purpose use – CAP_SYS_ADMIN grants the container a large number of dangerous privileges – this should be used only by a sysadmin (Ops) – not a developer – for the purpose of containerizing infrastructure.

Some of my insights gained on running systemd within docker – I’m aware the general idea is to run a single process, but that’s for developers. I’m a sysadmin, so I know that underlying “docker” as a management system is a full featured Linux namespace/cgroup facility allowing me to run a fully containerized Linux userland.

This is cobbled together from some of the example Dockerfiles provided by Red Hat’s Project Atomic as well as personal research into systemd, unprivileged container limitations, and the Linux capability system. This isn’t guaranteed to work on anything but the latest version of Docker at the time of this post – 1.4.1. May work on a previous version if it has support for –add-cap.

FROM centos:centos7
MAINTAINER "Brad Laue" <brad@brad-x.com>
RUN yum -y update; yum clean all \
yum -y swap -- remove fakesystemd -- install systemd systemd-libs \
(cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
RUN systemctl mask getty.target swap.target
RUN systemctl disable graphical.target; systemctl enable multi-user.target
VOLUME ["/sys/fs/cgroup"]
VOLUME ["/run"]
VOLUME ["/export"]
ENV container=lxc
CMD ["/usr/lib/systemd/systemd"]

Build with:

docker build --rm -t bradlaue/centos-systemd .

Run with:

docker run -ti --cap-add=SYS_ADMIN -e container=lxc bradlaue/centos-systemd

Note this does not require a fully privileged container instance as many seem to indicate – systemd requires only CAP_SYS_ADMIN in order to avoid a segfault.

From here you can build out a container by installing/running standard Linux services in the normal systemd way, including process monitoring.

Fun with VPS’s – brad-x moves into the cloud

In an effort to do some finance management I’ve switched from a dedicated server at $132/month USD to a VPS hosted by Burst.NET for the eminently reasonable sum of $10/month.

For this price I get 50GB of disk space, 1GB of RAM and 1.5GHz of allocated CPU clock – more than I’ll need!

I’m also working on bringing the screenshot gallery up to date with a comment system, tagging and social network connectivity. No, I won’t let it die already. 😛

Practically Replacing Microsoft Exchange Server – A 3 Part Series – 3 of 3 – Kerio Mailserver

Kerio MailServer 6.5 – The Exchange Killer

Kerio MailServer, like Zimbra, has until only recently been an ‘almost but not quite’ Exchange alternative. It has offered Outlook support and integration with Active Directory since 2002, but did not initially support groupware features such as calendaring and shared contacts properly until years later. It wasn’t until 2007 that Kerio began to coalesce into an alternative to Exchange — and with the release of Kerio MailServer 6.5, its transformation into an Exchange killer is complete.

For the first two parts of this review series, please view the following links:

Practically Replacing Microsoft Exchange Server – A 3 Part Series – 1 of 3

Practically Replacing Microsoft Exchange Server – A 3 Part Series – 2 of 3 – Zimbra Collaboration Suite

Kerio MailServer 6.5 – The Exchange Killer

Kerio MailServer, like Zimbra, has until only recently been an ‘almost but not quite’ Exchange alternative. It has offered Outlook support and integration with Active Directory since 2002, but did not initially support groupware features such as calendaring and shared contacts properly until years later. It wasn’t until 2007 that Kerio began to coalesce into an alternative to Exchange — and with the release of Kerio MailServer 6.5, its transformation into an Exchange killer is complete.

I’ve saved the best for last with Kerio — I prefer it over Zimbra as an Exchange replacement for several key reasons which I’ll outline below.

Client Software Compatibility

Kerio provides connectors for Outlook 2003 and 2007 which enable those clients, previously mentioned as irreplaceable tools for office workers, to work with Kerio as if it were Exchange itself. These connectors use the HTTP/HTTPS protocol, and as such a user can fully access their public folders and the global address list while working remotely as though they were in the office. This Outlook connector is provided at no extra charge.

Mac users are able to sync to Kerio through the use of the Kerio iSync connector, also provided at no extra charge. This connector provides both addressbook and calendar sync for users of MacOS X 10.4 Tiger, and addressbook sync for Leopard users (calendar sync can be natively accomplished by Leopard’s version of iCal, so the iSync conduit is not needed for this — though it can still be used).

Kerio also supports International standards such as CalDAV, enabling clients such as Apple iCal, Mozilla Sunbird, Novell Evolution and OSAF Chandler to connect with its calendar and participate fully with Windows/Outlook users.

Like Zimbra, Kerio has a rich web UI though it’s patterned closely after that of Exchange. In addition to this, Kerio takes it a step further with full emulation of Outlook Web Access, which is of benefit to any third party tool  or application that interfaces with Exchange via this mechanism. This opens up a larger segment of the Microsoft-entwined ecosystem to Kerio switchers than any other Exchange alternative.

Mobile Devices

ActiveSync is one of the most important features of Kerio. This isn’t emulation or the implementation of similar functionality via a third party app, it’s native, true ActiveSync protocol support. That means any ActiveSync device including Palm, Windows Mobile and the iPhone can sync all their information to Kerio with “Push” (instant notification) support and GAL search.

Blackberry users are also covered – an app installed on the handheld will enable push calendar/e-mail/contact synchronization without the need for a Blackberry Enterprise Server (BES).

Support for IT Infrastructure

The most prominent reason I prefer Kerio is that, despite its complex functionality it maintains utmost simplicity for systems administrators. Backing up and restoring or redeploying a Kerio mailserver can be done effortlessly, even when changing the host operating system, simply by copying it’s store directory as well as a handful of configuration files to the new server and then starting it. Email, contact and calendar data are stored on the filesystem rather than being placed in a database or needing to be specially imported and indexed.

The Linux version of Kerio, while officially supported only under Red Hat Enterprise, can be easily deployed on any modern distribution with little effort. This is in contrast with Zimbra which requires pretty major surgery to get running on anything other than its short list of supported distributions.

Kerio integrates with both Active Directory, supporting Windows networks, and Open Directory, supporting MacOS driven networks for authentication information, relieving admins of the need to maintain a separate user database. One nice thing about Kerio is that it can join multiple Active Directories on a per-domain basis, making it possible to host multiple mailsystems and multiple Global Address Lists on a single server.

If a single server is not enough, Kerio also supports clustering, and because its Linux and OS X versions support a wide range of UNIX filesystems and filesystem abstraction mechanisms, the mail spools and stores can be placed on a wide range of possible storage systems. Scalability is no problem.

Summing up

Kerio doesn’t come in a free version as does Zimbra, but this didn’t deter me from buying it for my own personal use. The benefits vis a vis Exchange (which I was previously using for my Calendar/Contacts/Mobile sync) were too compelling to pass up.

Kerio presents itself as a drop-in Exchange replacement, requiring as little re-training on the part of users and systems administrators alike (though system administrators should always be re-training themselves, a little elegance on the software side never hurt anyone). Some research has led me to find a growing number of Hosted Microsoft Exchange providers beginning to offer Hosted Kerio as well, which is an encouraging sign that it’s being recognized for its capabilities. I hope to see Kerio, Zimbra and others continue to take the de-facto center-seat away from Exchange in as many organizations as possible.

Practically Replacing Microsoft Exchange Server – A 3 Part Series – 2 of 3 – Zimbra Collaboration Suite

For a long time, Zimbra has been an ‘almost but not quite’ Exchange alternative – it offered the web GUI and the Outlook compatibility, but not the standards based calendar protocol (CalDAV) or the mobile device support of its big brother. Zimbra’s latest version, which has only been in the wild for a few months, is different.

 

Zimbra Collaboration Suite – A Full Exchange Replacement

For a long time, Zimbra has been an ‘almost but not quite’ Exchange alternative – it offered the web GUI and the Outlook compatibility, but not the standards based calendar protocol (CalDAV) or the mobile device support of its big brother. Zimbra’s latest version, which has only been in the wild for a few months, is different.

Client Software Compatibility

Zimbra provides an Outlook connector enabling Windows users to continue using the app they were trained on. All features in outlook down to shared folders and the GAL are provided. On top of this, in its latest 5.0 version, it has introduced wide ranging mobile device support, covering all major smartphones with the exception currently of the iPhone (though a web interface is provided specifically for iPhone users, this isn’t quite the same as total integration).

A connector for Apple Sync Services is provided, enabling Mac users to connect the native Address Book and iCal applications to tue server with full functionality and interoperability with their Outlook using counterparts.

Zimbra also advertises its mail and calendar facilities in a standard way so that a growing number of standards based clients can also access this information. CalDAV clients such as Apple iCal 3.0 (in MacOS X 10.5), Novell Evolution and Mozilla Sunbird are leading this increase in awareness of a standard calendaring protocol, and Zimbra does well to support it.

Mobile Devices

Most notably, Zimbra 5.0 introduced the dealmaker: support for PalmOS and Windows Mobile smartphones via an installable application, and blackberry support via an extension to BlackBerry’s BES server. This finally puts Zimbra on the radar for corporate deployment.

Support for IT Infrastructure

On top of all this, Zimbra supports largescale clustering and a massive number of clustered storage/backup options available to it due to its reliance on Linux and MacOS X as the server operating system of choice, and integration with Active Directory – IT departments everywhere will be able to work Zimbra into their infrastructure.

With the 5.0 release, Zimbra Collaboration Suite meets these criteria and has become the first answer to Microsoft Exchange.

Yahoo Buyout Attempts – The Empire Strikes Back

In September 2007, Zimbra’s parent company was purchased by Yahoo! and Microsoft began making hostile moves toward buying out  Yahoo! shortly thereafter. One has to wonder what Microsoft would have done to Yahoo!‘s Zimbra division if this had taken place – it’s unlikely Zimbra would have survived (Microsoft is famous for using phrases like “knife the baby” in its business deals). Had Yahoo! not rejected these attempts, there may not have been any alternative at all.

Fortunately, Yahoo! has so far rejected all attempts at being bought by Microsoft. Let’s hope this continues, because I find it doubtful that the US DoJ would notice the destruction of a potential Exchange competitor.

That is until…

Stay tuned for part 3, which will cover the second of these: Kerio MailServer 6.5.

Home movies with Linux: Not Ready for Prime-Time

UPDATE: I had originally titled this article “Home movies with Linux: Beginning the exploration (of things Linux can’t do)”, but have since changed the title to be a bit more fair. It is a given that Linux isn’t suited to certain tasks, but rather than show these things up as embarrassing shortcomings I’d much prefer to demonstrate these areas of weakness to people who strongly believe Linux should replace everyone’s Mac / PC tomorrow. It can’t, and shouldn’t. We should all use our heads to: a) find the best tool for the job, b) recognize and put work into fixing shortcomings in our own tools.

Most computers now ship with a way to create and save videos. Youtube’s prevalence is an indication of how readily people want to take a movie clip, digitize it, and post it for others to see.

Criteria:

  • Installation of the OS and the app have to be as simple as they are on commercial operating systems.
  • The app has to take video in any format, arrange clips and save the result. I won’t even ask for special effects.
  • Preferably export to DVD.

After some browsing of gnomefiles.org and the ‘Add/Remove Applications’ panel, it appears a program called ‘Pitivi’ is most suited to this task. I decided I’d check it out from a fresh install of Ubuntu Feisty, as that distribution packages and distributes it. The following are notes on the installation and use of both Ubuntu and Pitivi itself from a usability and quality assurance testing perspective.

The Installation

Ubuntu Feisty (7.04) install went well. Asked me my name, country, and the current time. Installation completed in approx. 20 minutes.

Nicely done. Up there with the major operating systems so far.

Updates and Installation of Apps

  1. After install, ran updates. During download (not installation phase) of updates, changed themes from Ubuntu Human to Clearlooks. Update manager crashed without giving an error.
  2. Opened Add/Remove Applications in order to add software. Selected ‘Totem movie player (xine backend) and ‘mplayer-plugin’, which is used to play video embedded in web pages. Was greeted with the following:

     

    ubuntu conflict
    Jigga what?

  3. Moved on to select Pitivi, the video editor. Was greeted with another error:

     

    Ubuntu package conflict
    Is this going to happen every time? I don’t think my mom could use this to make a home movie.

  4. Taking the suggestion of these error windows, I closed Add/Remove Applications and switched to Synaptic. Was greeted by the following high-quality user-interface:

     

    synaptic
    Looking great. Hey, I know, why don’t we collapse all the information panes so they’re virtually meaningless! Oh! Already done! Thanks guys.

  5. Expanded the synaptic window and clicked search. Searched for and checkmarked the three items I wanted and clicked apply. Installation proceeded without issue. Closed synaptic.
  6. Used Places > Connect to Server… option to connect to another computer to upload screenshots. This worked once, but after disconnecting from this share I was unable to reconnect to it. I received no error. Even as an advanced user I was unable to resolve this and had to resort to SCP from Gnome Terminal. Eventually the desktop froze requiring a user to restart the computer as there is no way of logging out in this situation (menus were unresponsive) and shouldn’t be expected to know about ctrl-alt-backspace.(The correct fix for this is: ‘killall nautilus’ which I used, and again, an end-user wouldn’t know).

     

    Jammed
    Nice.

  7. Pitivi Itself

  8. Pitivi was unable to understand MPEG-4/H.264 files. Dragging these and other types of files onto the Pitivi sources list resulted in no error, but no import either. Documentation indicated that it should work. Ubuntu did not install Gstreamer components necessary to facilitate this. I installed these myself using synaptic.
  9. Synaptic remembered the window was maximized last time it was launched. Gold star.
  10. Installed: gstreamer-plugins-ffmpeg, gstreamer-plugins-bad, gstreamer-plugins-ugly. I only knew to do this due to prior experience. No obvious cues pointed out that these packages were needed. An end-user would, at this point, be completely lost.Import now worked. Audio playback, not so much. As it turns out I needed the multiverse variant of gstreamer-plugins-bad and gstreamer-plugins-ugly. I wouldn’t have known this if I hadn’t googled.
  11. During test playback of a 320×240 H.264/AAC movie, Totem did not properly antialias the expanded video. I don’t care which subcomponent is at fault, it looks like shit. Also note the green artifacts on the left. That isn’t present in the video stream.

     

    Totem antialiasing
    Totem botching up playback.

     

    quicktime-antialias
    QuickTime’s rendering of the same frame. Perceptually superior, no blockiness during playback.

  12. Pitivi continued to be unable to import .mp4 files by dropping the files on the ‘dragging them here’ area. I had to click ‘Add’ before they would import.
  13. Pitivi did not present any obvious way to re-export the imported video. It presented limited format options, restricting me to AVI, OGG, or MPEG container formats and AC3 or MP2 audio+MPEG-4 or Theora video. The ‘New’ ‘Open’ ‘Save’ and ‘Save as’ buttons and menu items never became active. Browsing the Pitivi wiki did not reveal a solution (or even cover the act of saving a project).

Pitivi documentation and setup is impractical for real-world use. While the user interface of the app itself represents a good start, it’s little more than a drawing on a napkin. While shipped with Ubuntu as an “easily installable” solution for video editing, the documentation for Pitivi clearly states that it can’t do the following:

  • Sources
    • Moving
    • Removing
    • Resizing
    • Cutting
  • Effects
    • Transitions
    • Video Effects
    • Audio Effects
    • Application-level plugins

Summary

Further research required on Linux-based positional video track editing/muxing. There appears to be no solution for this need as yet.

As for Ubuntu itself, little thought is being put into shipping applications in a way the user can easily install. All visual cues point to Add/Remove Applications, which doesn’t work. Synaptic’s user interface is abominable. Dependencies are not properly installed, resulting in the need for expert intervention. The desktop environment can’t reliably perform tasks such as copying files to another computer. On top of this, Ubuntu is shipping apps that aren’t even fully functional yet. What good is a video editor that can’t move/remove/resize/cut video?

No Linux reality-check article is complete without a humiliating comparison with software designed by paid professionals. For an example of an application that _comes with_ a commercial operating system and allows you to make movies with sophisticated effects and import/export options, check out iMovie – Making Movies/Creating a movie.

Just because you’re unhappy with Windows doesn’t mean you have to go running off to something even less functional. Remember, choosing an alternative to Windows isn’t about hype, it’s about making smart choices.