How we got Linux on Windows
From "Linux is a cancer" to Windows Subsystem for Linux.
Mixed paints (source: stux via Pixabay)
Since the early 1990s, when Windows became much more popular in the enterprise, people have been trying to put Unix and Linux into places where it doesn’t want to be, using toolkits that implement just enough of the Portable Operating System Interface (POSIX) standard to feel like Unix.
The reasons are pretty simple: a lot of open source tools, especially development tools, are primarily targeted to Unix/Linux platforms. Although the most important ones have been ported to Windows, they are designed to work best when they have access to a Unix/Linux shell scripting environment and the many utilities that come with those platforms.
Learn faster. Dig deeper. See farther.
Today, there are many options availability for getting Unix or Linux functionality on Windows, and the most recent one, the Windows Subsystem for Linux (WSL), provides a Linux environment that is deeply integrated with Windows. But before I get into that, I’ll look at the some of the other options.
The early days
In the early 1990s, you could get a pretty good Unix shell, compiler, and text utilities on DOS using DJGPP. It was also around this time that you could simply install Linux on a computer if you wanted a pure Unix-like environment. Linux had none of the limitations that a Unix-like subsystem had, but it also meant that you had to convert in total to Linux.
So, if you wanted—or needed—to run a Unix-like environment alongside a Microsoft operating system, you needed a subsystem like DJGPP. And in order to comply with US Federal Information Processing Standards and be considered for defense-related projects, Microsoft needed one, too.
Windows NT, Microsoft’s first foray into true multitasking and multi-user operating systems, is the basis of their current operating system offerings, whether you’re running it on a phone, computer, or Raspberry Pi. Although it shares superficial traits with Unix and Linux, internally, it is not at all like them, which wasn’t surprising considering when Windows NT was born: in 1993, Unix was something you’d find on a workstation or server. Apple’s Macs were running their venerable System 7, and it would be eight more years before Mac OS X would come out, itself a Unix-based operating system.
The difference between Windows and Unix meant that when you ported a Unix application to Windows, there was substantial functionality, such as the fork() system call, that was simply not available. Because of this, the number of Unix programs that could be fully ported to Windows was fairly small, and many programs had reduced functionality as a result.
Over the years, Microsoft kept Unix and Linux at arm’s length. To comply with those federal standards, it supplied a bare minimum POSIX subsystem, just enough to satisfy the standards. The POSIX subsystem was eventually replaced with something called the Windows Services for Unix, which provided a Unix shell, a compiler, and a lot of command-line tools. But it wasn’t enough.
The age of Linux
It didn’t take long for Linux to define what the Unix-like experience should be on a PC. Full access to networking, a GUI, but also, an enormous library of software available for you to install. Out of the box, an Ubuntu system has more than 50,000 packages available to install. Do you need a simulator for electronic circuits? There are several available. How about a cross-compiler for MIPS, ARM, or PowerPC? There’s that, and more: an incredible number of programming languages, specialized software for ham radio, thousands of tools for manipulating text. Much of what Linux has to offer is enabled by the GNU project, which created the compiler, shell, and the most common utilities you’ll use day-to-day.
So it wasn’t surprising that the go-to Unix environment for Windows would look a lot like Linux. Cygwin, a Unix-like subsystem for Windows, implements enough of the POSIX API that it can include thousands of software packages ported from the Unix and Linux universe. Cygwin provides a shell, all the tools and compilers you’d expect from a Linux system, and implements the Unix (POSIX) system calls through a Windows DLL. If you run a Windows executable from within the Cygwin shell, it will run just like any other Windows program. But if you run an executable that was compiled for Cygwin, it will call into cygwin1.dll to provide the POSIX APIs.
In 2016, two unusual things happened that changed the relationship between Microsoft and the Unix/Linux world.
Windows and Linux, closer than ever
First, Microsoft partnered with Canonical, the makers of Ubuntu, to bring a full Ubuntu subsystem to Windows 10, the Windows Subsystem for Linux (WSL) beta. Its point of entry is the bash shell. After you enable the subsystem, run the bash command from Windows and you’ll be dropped into a bash shell. It doesn’t look all that different from Cygwin or from Linux or Mac OS X, for that matter. The difference is that from the perspective of a program that was compiled to run under x86 Linux, WSL is indistinguishable from Linux. In terms of the out-of-box experience, WSL may seem indistinguishable from Ubuntu 16.04 Xenial. But there’s a lot more going on. First, WSL is really a new infrastructure for Windows that implements a Linux kernel-compatible ABI (Application Binary Interface). It allows Windows to run unmodified ELF-64 binaries, and is distribution-agnostic. When you configure it, WSL downloads and runs an (unmodified!) Ubuntu cloud image, but WSL is designed to support other Linux distributions as well. There are a few system calls not yet implemented, but that list gets smaller and smaller with each Insider Preview Release of Windows.
Second, Microsoft became a Platinum member of the Linux Foundation. Fifteen years ago, Microsoft’s position was that Linux is a cancer. A lot has changed inside Microsoft. A lot has changed in the world outside Microsoft, as well.
Today, Unix and Unix-like are everywhere: Apple’s phones, tablets, and computers are all based on a Unix operating system. Linux has essentially come to define the modern Unix experience, and both Android and the Raspberry Pi operating systems are running a Linux kernel. Why Unix and Linux? First of all, Linux is free, so it’s an easy choice for anyone making a new device (Apple’s choice of Unix is historical; the OPENSTEP operating system, on which macOS is based, was a Unix variant). Second, because so many tools and applications are available (also free) for Linux and Unix, there aren’t as many wheels that need to be reinvented.
Now, Microsoft has a bash shell. Because of the deep integration with Ubuntu, it uses the same package repositories as any other machine running Ubuntu Xenial. Most of the applications in those packages will run just fine on Windows.
Years ago, when Apple chose Unix as its foundation for Mac OS X, we came out with Mac OS X for Unix Geeks, and some 60,000 readers used it to figure out exactly how Apple had rearranged their familiar environment. Over the years, Mac OS X’s Unix became a lot closer to Linux, adopting many of the same core libraries and utilities. And as Mac OS X matured, it’s now the norm for open source software to compile without all sorts of strange workarounds.
It’s a little different for WSL. From day one, WSL is Linux. Sure, there are some differences. For example, WSL doesn’t include an X server. But that’s a small matter of installing any one of the excellent X servers available for Windows, and setting your DISPLAY environment variable appropriately. Getting an X server running with WSL opens up the full Linux experience, so it should be one of the first things you do. I use and recommend the free and open source VcXsrv X Server.
If you’re a Windows user who needs to learn WSL, you can turn to the same resources that have been serving Linux (and Unix) users for years: If you’re completely new to the Unix/Linux shell, our free report “Ten Steps to Linux Survival: Essentials for Navigating the Bash Jungle” will get you on your feet. And when you’re ready for more, our Bash Cookbook includes 15 recipes just on things that are likely to trip up Bash novices, as well as dozens of recipes on sorting, parsing, and automating day-to-day tasks. If you’ve got some familiarity with the Bash shell or Unix/Linux in general, Bash Pocket Reference, 2nd Edition, can help you find what you need when you need it.
As betas go, WSL is incredibly well-baked. If you’re on the Windows insider program, you’ll benefit from frequent updates to WSL, and you can monitor progress on their GitHub issue tracker, which is full of feature requests, discussions, and the hopes and dreams of everyone who wants a little bit of Linux in their Windows. The high water mark I’m waiting for is if Docker will run natively on WSL. That would present an attractive alternative to the Docker on Windows solutions that depend on Hyper-V or VirtualBox and would be a huge step forward in resource utilization. Here’s to hoping we get there in 2017.