Likewise Blog

Many Linux users are familiar with sudo these days. Ubuntu has done a lot to popularize sudo by enforcing its use in place of encouraging users to use su to switch to the root account to install software and perform other administrative tasks. But there’s much more to sudo that users and admins should know.

What many users aren’t aware of is that sudo can be used to execute commands as any user, not just the root user. In the hands of a skilled admin, sudo can be used to set up fine-grained permissions to provide users with access to perform a few administrative tasks without giving away the keys to the kingdom. Let’s look at some of the best practices for controlling system access with sudo while still allowing users to be productive.

Read the rest on Linux Planet here:
(http://www.linuxplanet.com/linuxplanet/tutorials/7165/1/)

I just finished reading Mike Vizard’s excellent post, “Authentication in the Cloud” and I thought it would be worthwhile to go into more depth about the issues regarding authentication and cloud computing. While the topic seems pretty straightforward (“validate my username/password”), it gets incredibly messy very quickly.

Before I talk about authentication in the cloud, let me review something more mundane: authentication on your private network. What happens when you login to your machine and then try to access some resource, say a file server or database, on your network? Yes, of course, when you login to your computer, something needs to assure that your username and password are valid.

If you’re logging into a Windows machine, this authentication is performed by a component called the “Local Security Authority Subsystem Service”. If you run Windows Task Manager and list the running processes for all users, you will see a program called “lsass.exe” – that’s the one. By the way, if you run Likewise on a Linux/UNIX/Mac machine, you will notice that we add a daemon called “lsassd” – guess what that does?

LSASS (Microsoft’s or ours) can authenticate a user one of two ways: using “local” credentials or using Active Directory credentials. If your machine is “joined” to Active Directory, you will typically login to your machine with your AD account (including the appropriate “domain” name). If your machine is not joined to Active Directory (it’s in “workgroup” mode), you will login to the machine using local credentials. In the local case, your username and password are validated against account information stored on your own machine. In the AD case, however, something more significant happens: LSASS authenticates your credentials using the Kerberos protocol to talk to an AD domain controller.

Kerberos is a wonderful thing. It can authenticate credentials without ever transmitting a password in either clear or hashed form. This is important because it makes it impossible to perform offline password cracking (i.e. trying millions of passwords until the cracking code matches your hashed password). Kerberos is also great because it supports single sign-on. Once you are logged in to your machine, you have a special “ticket” that can be used to acquire additional tickets for other services. Once you are logged on to your machine, if you go and access a Windows file server, the file server will not prompt you for credentials if your logon credentials are sufficient. Under the cover, the authentication code automatically acquires a “service” ticket for the file server based on your logon ticket. If you access a SQL Server database or a Microsoft IIS-protected web site, again, you don’t need to enter additional credentials because the necessary service tickets are automatically acquired. Nifty.

If you logged in using local credentials, you don’t get any of this goodness. When you try to access a file server, it will perform older “NTLM” authentication and realize that it doesn’t know anything about your local account – if the files on the server are protected, you will be prompted for credentials in order to access them. With SQL Server and with IIS you’ll need to use more primitive authentication techniques (“SQL authentication” or basic authentication, for example).

Challenges of Authentication in the Cloud

Consider now “the cloud.” Or, rather, clouds, because you’ll find several offerings that fit under the cloud umbrella.

Well, first consider that the initial cloud that any of us will encounter is really “the local VMware datacenter” or as it’s now being called, “the Private Cloud.” No problems with the Private Cloud – it’s connected to our network and can fully participate in the goodness of Active Directory and Kerberos.

The second cloud we’ll encounter is “the Hybrid Cloud” – an external cloud of computing resources that serves as an extension of the internal datacenter allowing workloads to be offloaded as needed. How authentication takes place in the Hybrid Cloud depends on exactly how its implemented. Many in the cloud business, however, suggest that it should be connected to the Private Cloud via virtual private network (VPN). If this is the case, the external cloud resources are effectively part of the internal network and we’re back to the simple Private Cloud scenario.

So, finally, let’s consider the “Public Cloud.” This is where things get messy.

The Public Cloud is generally defined as being external to the companies that use it. It is where SaaS-type applications (typically web-based) will run.  Microsoft’s Azure platform, for example, defines a very compelling set of libraries and software architecture that simplify the creation of massively scalable apps. “Let a hundred [web app] flowers bloom”, to misquote Mao Zedong. How does authentication work in this context? When you login to a web application, how are your credentials validated? The answer is, unfortunately, “all kinds of ways” – LDAP, database lookups, file lookups, even Kerberos, sometimes.

Tower of Babel in the Clouds

The problem with this lack of cohesive authentication strategy is that it makes it difficult to effect things that we take for granted in the Private Cloud. When somebody leaves the company, how do we disable his/her user accounts on outside SaaS apps? How do I keep track of my passwords for all my various applications, since they don’t support single sign-on? How can we enforce password policies when no two systems use the same authentication mechanism?

We are just beginning to solve some of these problems.  SAML (Security Assertion Markup Language) and ADFS (Active Directory Federation Services) are protocols, for example, that allow “identity federation.” They enable apps that run on the Public Cloud to authenticate users with their own corporate credentials. SalesForce, for example, allows you to login with your own corporate username and password if you set up a federation agreement with them. Unfortunately, identity federation is well defined only for some simple web application use-cases. There’s no easy way for me to back up my local disk to a Public Cloud service automatically using my corporate credentials to authenticate me to the service. I can’t write a local application that accesses a SQL database on a Public Cloud that enforces security using my federated corporate identity.

Ultimately, we need to remove the distinction between corporate and web authentication – all authentication should be based on Internet-routable protocols and the nature of identity federation should become simpler (there should be no need for internal/external identity federation servers).

This will take a long time – Microsoft’s version of LSASS knows nothing about web-based authentication protocols. When we do figure this out, however, we should make sure that we do not lose the many benefits of our current mechanisms. Kerberos is a great security protocol, offering the user much convenience. Let’s figure out how to more broadly apply it in the Public Cloud context.

I was reading a blog a couple of days ago about a company’s security manager being left out of the loop when the company reorganized its Active Directory architecture — and a couple of thoughts occurred to me that I thought I’d share. The post recounts how his company made some directory topology decisions without involving him, the Security Manager for the company.

Now, I don’t know  the writer and it’s possible that he’s a superb Security Manager and that his company made a serious oversight in not consulting him regarding these decisions. I think it’s also possible, however, that the company deliberately chose to exclude him from these deliberations. Why would they do this? Maybe he doesn’t understand the issues. Maybe he’s so security conscious that he’s obstructive. Maybe he’s too slow at making decisions.

At Likewise, we frequently have to deal with Security Managers or with CSO’s (Chief Security Officers) and we’ve run into both good and bad ones. Security is a difficult topic. Understanding it requires familiarity with high-level policy and regulatory issues (e.g. HIPAA, SOX, etc.) and with low-level details (ACLs, roles, etc.). It’s hard to find security personnel who have mastery over the entire gamut of issues. It’s also hard to find people who have the right balance of risk and reward. It’s easy to ensure security simply preventing everyone from doing anything risky. A good Security manager has to have both the right technical chops and the right balance of risk and reward.

As to the specific technical issues in the blog post, the Security Manager suggests that his company’s decision to go with a single-forest/single-domain Active Directory design sounds wrong to him that, perhaps, it would be better to go with single-forest/multiple-domains. He doesn’t explain why he thinks this is better. It’s possible that he’s right, but I would make the following observation: our largest customers tend to have the simplest AD topologies. Many ended up with simple topologies after suffering through more complex ones. Although child domains are simpler than inter-forest trusts, they still add complexity to the design and sometimes result in LDAP referral messages that can trip up applications. He’s certainly right about other issues however, for example, the dangers of centralized v. distributed domain controllers.

So, my advice to  the writer is that when your girlfriend tells you “it’s not about you, it’s about ME”, that it sometimes really is about you. But then, again, I don’t know him.

I’m often asked why Likewise invests in Likewise Open. Clearly the target market for Likewise is the enterprise market, yet we work hard to provide a set of core functionality under GPL/LGPL that allows any individual or organization to join machines to an Active Directory network for free. Moreover, that code would allow anyone to modify, extend, and ship the same functionality in a community project or commercial offering. How does that benefit Likewise?

The benefits of having an open source offering are not always immediately obvious. However, we’re in it for the long haul and see that the long-term benefits of being open source are good for Likewise, our customers, and the open source community at large.

Helping the Open Source Community

Some of the benefit that we provide to the community could also be derived from offering a proprietary but free-as-in-beer offering. But there are a few things that wouldn’t be possible if we only provided binaries.

First of all, providing source under an open license means that we can ship with Linux distributions like Ubuntu as part of the main distribution or in the regular open source repositories. This is a huge deciding factor for us, because we want to allow any and all Linux users to be able to interoperate with an Active Directory network regardless of whether their company is a Likewise customer. Many of our community members would be faced with using Windows laptops or workstations if they weren’t able to authenticate against AD, regardless of their personal preference. That’s not desirable at all.

We want to make sure that Linux, UNIX, and Mac OS X are first-class citizens in any network, and the reality is that many organizations depend on Microsoft Active Directory for authentication and user management.

We also want to enable the community to package and ship Likewise Open with any Linux distribution. We obviously can’t package Likewise Open for every distribution ourselves. But any distribution that wants to offer an RPM, Debian Package, etc, can do so without needing to ask permission from Likewise.

Another consideration is that an open source stack allows users and distributions to compile Likewise Open for other platforms if they choose. Most Linux users are running on x86 or x86_64 systems, but we’d be pleased to see Likewise Open running on PowerPC, ARM, you name it. These platforms may not be strategic to Likewise, but they’re important to many users. Being open means that the community can take Likewise Open to platforms we aren’t shipping on yet, and gives the community the ability to modify Likewise in ways we might not.

Benefits to the Customer

That’s all great, but how does that benefit Likewise’s customer base? For one thing, we learn a lot from our community about the core product in the form of bug reports, questions in the Likewise forums, and interacting with the larger community.

It also helps Likewise work with distribution vendors. Our partnerships with companies like Canonical and Novell are strengthened by the fact that they’re able to recommend Likewise Open to their communities, while offering Likewise Enterprise to their enterprise customers. Again, it’s very important that every user is able to join to their organization’s network, including users running community Linux distributions like Ubuntu, Fedora, openSUSE, Debian, and others.

And, of course, Likewise Open makes Active Directory authentication available to everyone. Whether it’s an Ubuntu-using developer in a mostly Microsoft shop, to a 50-seat art department using Apple machines inside a major corporation with a wholly heterogeneous network.

We also feel indebted to the open source community. The solutions that Likewise provides are largely in response to opportunities made possible by the Linux and open source community. We’re committed to sustaining that community and ensuring that every user that needs to operate in a heterogeneous network has the opportunity to do so. That’s why we’re open, and not just free.

Jay Lyman has hit the nail on the head on the 451 CAOS blog regarding the open core debate. The primary reason vendors are exploring open core as a model is because customers have asked for it.

With Likewise Open we provide the features needed by our entire community, from individual users to enterprise customers with thousands of seats. The primary functionality, connecting to and authenticating against Microsoft’s Active Directory, is fully open source and benefits everybody. We’re pleased to be able to offer Likewise Open under the LGPL/GPL, and to know that this is working for more than 50,000 customers around the world.

Our enterprise customers have the added value of Likewise Enterprise, which adds a number of features to Likewise Open, such as directory migration, reporting and auditing tools, and single sign-on for enterprise applications. The added functionality in Enterprise benefits a very specific segment of our community, and we work closely with our enterprise customers to ensure we provide value here. At the same time, we work hard at making Likewise Open a solution that is robust and updated simultaneously (if not ahead of) Likewise Enterprise.

Simon Phipps and other open source advocates are right to watch for companies that would use the term “open source” abusively or deceptively. But this does not include all open core strategies. Phipps does many of the vendors (and their customers) offering open core solutions an extreme disservice by painting open core as some kind of nefarious lock in scheme. The converse is true: Many open core vendors are providing functionality that benefits an enormous user base whether they do business with the vendor or not.

Open core, when done right, provides value to the open source community and consumers of the proprietary software simultaneously. It may not be the ultimate ideal for those like Phipps who spend their time criticizing open source businesses that don’t achieve their desired level of software licensing purity, but it’s a workable solution that addresses the needs of the customer, community, and vendor.

Virtualization has taken the one-to-one relationship of the physical server to operating system and turned it into a one-to-many relationship between the physical server and multiple guest operating systems running on the hypervisor.  As the management frameworks for running these guest operating systems within the hypervisor mature, creating in essence the “virtualization layer,” we are seeing an increasing amount of choice as out-of-the-box virtual appliances and purpose-built virtual servers catch fire in the market.

What is key here is the shift to a focus on the workload running on the guest OS – not the underlying OS.  Despite some vendor prognostications / protests, I predict the shift to a focus on the workload, meaning the applications / services the business cares about will continue and the OS will become more commoditized.  The key to success in this shift is meeting expectations of customers that the appliance or virtual server should just work with their existing legacy systems and processes.  If I spin up a virtual appliance, it must support seamless migration from (or integration with) legacy systems.

What is Likewise doing to support this shift?  Today Likewise provides platform interoperability for improved authentication and access control in virtualized environments for more than 75% of the hypervisor market (if you combine market-share for VMware and Citrix). Here’s what we do for each:

VMware embeds Likewise Identity Services to allow Microsoft Active Directory users to log-in to ESX/ESXi hosts.  VMware notes in a recent announcement, “As customers continue on the journey to cloud computing, they need to leverage existing security infrastructure for their virtualized environments,”

Citrix embeds Likewise Identity Services in XenServer, noting in a recent announcement, “It’s critical to provision users, applications and computing resources so that our virtualization technology has the same high levels of security as actual physical hardware.”

Likewise Enterprise then goes a step further, allowing users to manage authentication, access control, group policy, and reporting for demonstrating compliance for audit purposes across the physical and virtual environment, including the hypervisor and guest OS.  That end-to-end control allows organizations to address security–one of the main impediments to virtualization adoption.

With our focus on platform interoperability, we’re helping those progressing down the virtualization path focus on the workloads they are running, and doing our part to make the choice of what operating system on which to run a workload ubiquitous.

You could have easily missed this one.  VMware and Novell announced they are expanding their strategic partnership.

VMware and Novell Expand Strategic Partnership to Deliver and Support SUSE Linux Enterprise Server for VMware vSphere Environments

There is a much bigger story here on two fronts.  The first is that while the language in the release carefully avoids the word “free,” this is in fact how VMware’s vSphere customers will view this announcement.  vSphere licensees are now also entitled to a license of SUSE Enterprise Server (SLES)–once the crown jewel of the SUSE business.  For free.  And software patches and updates.  For free.  I’m not going to make any prognostications about how this may or may not change the landscape, but I’m pretty sure that large VMware shops are going to at least take a pretty hard look at SUSE.

More details on the “free” part of this announcement on VMware’s site:  SUSE Linux Enterprise Server for VMware

The second story playing out in front of us I believe is a larger one.  The value the OS has to IT operations has been waning over time.  Of course this has almost always been true on the business side of the shop–they care much more about applications for example and not very much about whether servers run Linux or Windows or Solaris.  But IT operations teams cared about operating systems because it was their foundation on top of which they built everything.  The foundation is changing. It isn’t the operating system–it is becoming the virtualization layer.  I believe the VMware / Novell announcement is another small step in the ongoing movement towards the complete commoditization of the operating system.

This announcement is also very relevant to our space.  As I’ve previously blogged, customers are increasingly looking for common IT services across the hypervisor, guest operating systems (Windows and Linux) and various other management and application elements.  Authentication is as often as not one of the first things that organizations want to unify across these layers and almost always this involves a mix of different operating environments.

Both VMware and Novell are Likewise partners and both are commercial licensees of various pieces of Likewise technology.  Novell, both through their partnership with Microsoft and through their licensing of Likewise Enterprise for SUSE Enterprise Desktop, has been aggressive in differentiating around Linux/Windows interoperability.  As we’ve previously announced, VMware licenses our authentication technology know as LWIS (Likewise Identity Services) which, when it ships shortly in vSphere, gives VMware customers the ability to manage privileged user access via Active Directory.  Add in Likewise and SUSE and you have common authentication in mixed Windows environments across the hypervisor, SUSE and Windows guest OS instances, and various other applications.  Very cool.

Congratulations to team Ubuntu who today shipped Lucid Lynx Desktop and Server editions.  As many of you know Likewise Open 5.4 ships as part of Lucid Lynx.  We’ve worked on a number of customer projects with the Canonical team and we continue to see growth of the Linux platform in all areas of the enterprise.

I frequently get asked about the state of affairs in the Linux world both relative to the platform vendors and relative to the degree of penetration of Linux as a desktop and server platform.

I’ll focus briefly on the Linux desktop.  There are three types of Linux desktop projects that I’ve observed:

The Technical workstation. Often engineers, sometimes former Solaris, HP/UX, and AIX users, have also spent time with Windows before moving to Linux.  What Linux variants?  We’ve seen it all:  RHEL, SLED, Ubuntu are all represented; we also see Debian, CentOS, and Fedora.  This group of users are almost always unabashedly enthusiastic about using Linux and have talked to me at length about the virtues of Linux as a viable desktop platform.

The Single Task Worker. Usage includes vertical markets such as call centers, manufacturing, and retail.  Usually multiple workers share a single computer and the security settings as managed by the company are fairly restrictive.  I have spoken to users on several of these projects that were unaware of the details of the underlying operating system.  Their interest was usually in the application that they use to accomplish their job instead of the platform itself.

The Information Worker. Linux is clearly not a mainstream platform for the information worker today.  There is, however, a non-trivial number of early adopter organizations that are experimenting with new platforms and hosted applications in a way that is quite different than the dominant paradigm today — Microsoft Windows + Office.  Many of the interesting projects that I’ve seen involve desktop virtualization.  Some organizations are experimenting with allowing users to choose whatever platform they want including Windows, Mac, and Linux while they provide a tightly controlled virtual desktop environment.  There are some very interesting intersections between the desktop, virtualization, and cloud computing.

Likewise gets involved with these projects because organizations want to leverage the infrastructure they have put in place to manage their Windows desktops.  Almost always this includes Active Directory both for authentication and as well as using group policy to enforce various security policies and settings.  With Likewise they can bridge their Linux and Mac users directly into Active Directory.

If I look to our customer base, we usually wind up with the Linux administrator as a big Likewise fan.  By using Likewise, they are able to offload authentication and account management issues to the same helpdesk that already supports the Windows desktop infrastructure.  We also allow them to integrate familiar Unix tools like Sudo into Active Directory which can be a powerful and highly scalable solution.  One of our customers told me this was “the best of both worlds” from her perspective.  From the users point-of-view, we help organizations take a big step in the direction of single sign-on which means less complexity for the user.

Professionally speaking, the past sixteen months have been, for me, some of the most exciting at Likewise that I can remember.  Today was the culmination of the combined effort of our entire engineering team as we announced that Likewise-CIFS is the integrated SMB/CIFS solution for Windows client support on some Hewlett-Packard StorageWorks products.

What makes this announcement special to me personally is not the fact that a major vendor has chosen Likewise; we’ve already established ourselves in the AD authentication bridge space with companies such as Isilon, DataDomain/EMC, VMware, and Citrix.   What makes the HP announcement particularly meaningful to me is my personal involvement in the HP partnership.  Over the past several months I’ve gone through all the daily status meetings, bug triages, late night debugging sessions, and the general things that go with enterprise scale software development.  Having worked at HP prior to coming to Likewise in 2005, I’m glad to see company endeavors succeed, particularly now that I have new friends amongst HP engineers.

Milestones like this announcement are good times to review where we’ve been and where we plan to move towards in the future.  In January 2009, Likewise began an initiative that would become Likewise-CIFS, the SMB/CIFS file server component of the Likewise Open project.  Even though I’ve worked on another SMB server in the past, Likewise-CIFS was really a brand new start.  The server’s multi-threaded architecture and modular components allowed us to parallelize much of the initial work, which was extremely important because the entire file server was being written from scratch.

To fully appreciate the difference between where we started in January 2009 and where we are now in April 2010, it’s best to examine the heart of Likewise Open–the code itself.  A quick glance at the repository from git://git.likewiseopen.org/likewise-open shows around 6800 commits.   That means that over the 337 working days (discounting weekends but including holidays) in the last 16 months the project has averaged about 20 commits per day.   Of course commits in and of themselves do not necessarily equate with improvement.  How many of the 6.8K commits added new code and new value?   Looking strictly at new components, we can conservatively say that a minimum of over 360,000 lines of new, handwritten C code has been added.  That doesn’t even include the 130,000 lines of C# code included in the Likewise Management Console that was made available under the LGPL last year.

If you aren’t a programmer, these numbers probably contain little meaning.  In that case, let’s talk about features.  In the past year, the following is a list of some of the new things that have been added to Likewise-CIFS:

• A complete SMBv1 implementation compatible with Microsoft Windows 2000 and later clients as well as Apple’s OS X 10.5+ and Linux desktops
• SMBv2.0 when serving Windows Vista/7/2008 clients
• Zero Copy support in the SMB server for faster reads and writes
• Scalability up to 50,000 connections on modest commodity hardware
• A GSSAPI NTLM mechanism for application developers desiring to integrate in Windows authentication frameworks
• A user space file system driver implementing Windows compatible byte range locks, opportunistic locks, share modes, security descriptors, and change notification.
• Server-side support for named pipes (necessary for integration with Windows RPC services)
• Privileged user management via the Likewise Security Authority’s (lsassd) local provider
• Managing all Likewise services from a central service control interface
• Central configuration management using the Likewise Registry

All this describes the road we have traveled thus far.  What about our future plans?  Will the next twelve months be as exciting as the past sixteen?  I believe so and here’s why.  There’s several what I call “point” features still remaining for the file server.  Things like Distributed File System (DFS) support and consolidation roots, Access Based Enumeration (ABE), Shadow Copies, and Alternative Data Streams (ADS) are isolated, individual features with a high degree of end user visibility.

But these are really just enhancements.  What broad initiatives do we have in play for the coming year that would match the scale of writing a new SMB file server from scratch?  We have several ideas already in discussion which I hope to be able to share in the coming months.  But sufficient to say that our path forward is to continue to build upon the Likewise Open platform base that we’ve put into play.  The way forward is up–to build upon the foundation already laid.

Interesting article on virtualization and the cloud by Amy Newman at ServerWatch.

http://www.serverwatch.com/article.php/3871306/Big-Blue-Bets-Big-on-Red-Hats-Virtualization-Technology

Aside from covering the news (Red Hat/IBM and VMware/Likewise announcements) Amy makes some prognostications of how the virtualization market may play out over time:

• VMware wins virtual server market in mid- and large-sized enterprises.
• Microsoft takes the top position in SMB virtualization.
• Citrix owns the virtual desktop.
• Red Hat succeeds in the cloud.

Interesting.  Especially interesting for Likewise, as we work with all four of these companies.  We license portions of our technology to both VMware and Citrix; we partner with both Red Hat and Microsoft.  We know all of these companies very well.

Increasingly I’ve heard our customers discuss their Likewise deployments in context with their cloud initiatives.  Here’s my take on the cloud, enterprise computing, and Likewise:

• The cloud is nearing the peak of the hype cycle.

• For the enterprise, most of the action (meaning $s) is in the internal cloud.

• The internal cloud is the modern data center — highly virtualized with the promise of vast scalability with a utility computing model.

• Companies are looking for common IT services across hypervisors, operating systems (Windows and Linux), and management components.

• The internal cloud is a bridge to hybrid and pure cloud topologies for enterprise computing.

• Winning technology providers will be those that unify these various pieces and platforms.

• Many organizations view authentication and policy as the right place to begin this unification process.

The one thing I know for sure is that the winner of the virtualization/cloud scrum will embrace cross-platform computing.  CIOs want to work with companies that make the various platforms work well together, not ones that make the problem worse.

Many of our customers tell me that Likewise is a great position for the cloud computing revolution.  I agree.