WhoAmI? The Java and Identity Management Blog

Announcement: KeyCenter4NB 0.1

2008-12-29T13:46:00.003-05:00

I'm very torn when it comes to choosing my Java development environment. I often swap between Eclipse and Netbeans. In fact, when I create a new project, I ensure that the project structure is compatible between the two, just in case the mood strikes me to swap back and forth. Eclipse is a fine IDE, however it is just a bit too rough around the edges in some respects for me. Netbeans, on the other hand, has some great features (UML, Matisse, a very powerful debugger and profiler, etc.) but I find it's Java editor very lacking and non-intuitive. The other problem I have with Netbeans is the lack of plugins. The Eclipse community has a far greater number of plugins available for it.

Recently, while working on some PKI-related code, I found myself wishing I had a plugin for viewing X.509 certificate details. After looking around, I really couldn't find anything suitable for my needs. I've never been really big on desktop GUI programming, mostly because I've never had the need to do it. All of my professional development has been on the J2EE side of things. However, I decided that if you want something done right you have to do it yourself, and decided I was going to write the plugin I was looking for.

After investigating both the Eclipse and Netbeans platforms, I decided to target Netbeans. I like the fact that Netbeans is pure Swing, and its GUI builder is second to none. I began playing around with module development, and was very pleasantly surprised with the experience. Yes, the Netbeans RCP has a very steep learning curve, and I've not even scratched the surface of what it is capable of. However, I've managed to create a very rough initial version of my plugin, KeyCenter4NB.

Initially, I was just looking for a plugin that would let me view the details of an X.509 certificate. However, the more I got into developing it, the more I wanted to eventually implement. As a result, the initial 0.1 version is just a certificate view. Future releases, though, will include tools for creating and manipulating Java keystores, creating PKCS #10 certificate requests, checking CRLs/OCSP, and other PKI-related functionality that a developer may need to take advantage of. A grand vision I'd like to implement (if I actually get enough time) would be a standalone application based on the Netbeans RCP that focuses on certificate and key management. However, that is a long way away.

In the meantime, check out the KeyCenter4NB project page and download version 0.1. Right now, you must download the .NBM packages and manually install them. I hope to have an update center for the project configured and running shortly. As I mentioned, it's very basic and I'm sure it's rough around the edges. But, I'll continue to update is as I learn more.

Syntax Highlighting in Blogger

2008-09-22T15:33:00.003-05:00

Recently, I've gone through and added syntax highlighting to some of my previous posts. This was done for Java, XML, and SQL. There is a great set of CSS files and JavaScript to accomplish this called SyntaxHighlighter. The downside, however, is that Blogger does not make integrating these tools very easy. I tried several times, but always managed to screw something up. As a result, I was very happy to come across a Blogger Widget for SyntaxHighlighter. All you need to do is add this widget to your blogger layout. Remember to leave the title blank so that you don't see an empty section in your sidebar. Once you've added this widget, you can add syntax highlighting similar to what you see here. To highlight Java, for example, edit the HTML of your post and surround your Java code with :


<pre name="code" class="java">
</pre>

That's all there is to it. A finished example would look like this, once highlighted:


public class MyClass {
   
   private int myInt = -1;

   public int getMyInt() {
     return this.myInt;
   }

   public void setMyInt(int myInt) {
     this.myInt = myInt;
   }
}

Intro to JBoss Seam Security, Part 3 -- Authorization Rules with Drools

2008-07-30T08:05:00.025-05:00

In previous articles, we have looked at how JBoss Seam handles security, particularly user authentication. Through the use of its Identity and Authenticator APIs, Seam provides an easy way to authenticate a prospective user without forcing a developer to implement hundreds of lines of code. In addition, Seam also provides CAPTCHA support, enabling an application to test for the presence of a human user, rather than a bot attempting to access an application. No doubt, Seam 2.1's forthcoming identity management API will make the process of creating and managing user accounts even easier. (Although, that is an article for another day)

While authentication is vital to application security, it is not the only aspect of security that must be accounted for. Authorization -- the process whereby it is determined what resources or actions a user may access -- takes authentication one step further, and cannot be overlooked. It is easy to come up with examples of where authorization should be determined within an application. A banking site would want to ensure that an authenticated user is, in fact, the owner of the bank account they are trying to access. A health insurance provider needs to check if an individual should be allowed to view recent claims that have been submitted. To accomplish this, some sort of authorization rules need to be established. Java EE already allows for programmtic authorization checks through the use of the HttpServletRequest.isUserInRole() method. Seam, however, takes authorization further and allows for integration within enterprise applications through the use of an authorization rules engine.

Seam uses the Drools business rules engine to allow complex authorization rules to exist outside of the Java code itself. Business rules engines have long been used to drive business operations or make decisions related to a specific business process (i.e. Approve a loan for an individual if they make more than $150,000 per year and have less than $25,000 in existing debt.) Now, these same types of rules can be applied to application security logic. Imagine a site like Facebook and how it approaches security. Facebook has very well-defined rules for how its information can be accessed. A profile is only visible to a member if the profile owner is friends with the member. Picture X can only be viewed by family members and friends, not coworkers. Member A can send a friend request to Member B provided that: 1) Member A != Member B, and 2) Member A is not already friends with Member B. It would be very easy to implement rules like this in Java code, either in the view or business layer. However, in our example application, we will separate these rules from our Java code and implement them using Drools.

Our simple example will be a social network site. Like any social network, ours will have some basic rules established to ensure the security of our site. The rules we will implement are:

Administrators may do whatever they please
A friend request may only be submitted if the sender != recipient
A profile is only visible by its owner and their friends

Our example will consist of two main objects:


public class Member implements Serializable {

   ...snip...

   public Set<Member>getFriends() {
      ...
   }

   public void setFriends(Set<Member> friends) {
      ...
   }
}

@Name(value="member")
public class MemberAction {

   ...snip...

   @In
   private Member authenticatedMember = null;

   public void viewProfile(Member member) {
      ...
   }

   public void sendFriendRequest(Member recipient) {
      ...
   }

   public void deleteProfile(Member member) {
      ...
   }
}

For the sake of time, we will not discuss basic dependency injection concepts. It should be obvious that MemberAction.authenticatedMember is related to the profile of a user who has successfully authenticated against the application. There are two primary ways we can perform an authorization check in the business layer using Seam: 1) Through annotations, or 2) Inline, using Java code. In addition, we can also perform security checks in the view layer to display or hide links or data based on the outcome of the check. We will discuss all of these methods for performing the checks as we move forward.

Before we are able to write and test our security rules, we must first enable them in our Seam configuration, and ensure our rules file is located in the right place. The components.xml file must have the following lines to tell Seam that we wish to use Drools for authorization:


<drools:rule-base name="securityRules">
 <drools:rule-files>
     <value>/META-INF/security.drl</value>
 </drools:rule-files>
</drools:rule-base>

As you can see, we have configured our security rules file to be located at /META-INF/security.drl. It is important that this file exists, otherwise our applications will not work as we expect. To tackle our first rule, "Administrators may do whatever they please", we need to establish a rule that allows administrators to be granted whatever permissions they request. Since this is a relatively simple rule, it will serve as a good starting point for our introduction into Drools syntax. Our rule will look like this:


package MySecurityRules;

import org.jboss.seam.security.PermissionCheck;
import org.jboss.seam.security.Role;

rule AdminsCanDoAnything
   when
      c: PermissionCheck(name == "member")
      Role(name == "admin")
   then
      c.grant();
end;

Let's take a look at the syntax for this rule. On line 1, you will see a package declaration. This is not the same as Java packages. Drools packages are used to group Drools rules together. They are not used for anything else. On lines 2 and 3, you see two import statements. This lets Drools know that we will be referrencing the specified classes as part of our rule. It is imporant to include these lines, otherwise our rule will fail. Line 4 marks the beginning of the rule itself. Our rule is called "AdminsCanDoAnything". There is no specific naming requirement for your rule, other than the fact that it must be unique within the given package. Typically, the rule name is a description of what the rule actually accomplishes. The rule is divided into 2 sections, the "when" and the "then". The "when" section consists of one or more conditions that must be true for this rule to fire. If any of these conditions is not met, the rule will not fire. Once all conditions have been met, the rules engine moves on to the "then" section, and performs whatever commands are specified.

Lines 6 and 7 provide the authorization check itself. Line 6 is the first condition that must be met for this rule to fire. The line:


c: PermissionCheck(name == "member")

can be explained as, "there must exist a PermissionCheck object with a name property of "member" within the current working memory. This object will be called 'c' in this rule." If this condition is met, the engine moves on to line 7. Line 7 specifies that, "there must exist a Role with the name 'admin' within the current memory." By now you're probably wondering, "What is 'the current memory'?" Drools has its own context or working memory that it uses when evaluating a rule. It essentially acts as a session for the rules engine. Before an object can be considered within a rule, it must be added to the working memory. Whenever a permission check is performed via the hasPermission() method, a PermissionCheck object is created and inserted into the working memory. This object contains the name of the object the check corresponds to, and the action that is being attempted. In our example, the name of the object is "member". You'll notice that we have not specified an action in our check. This results in the rule corresponding to all possible actions pertaining to member. In addition, any Roles that a user belongs to will be inserted into the working memory prior to a rule check. This is how we determine the contents of our example security rule. It is also possible to insert an arbitrary object into the working memory by calling


((RuleBasedIdentity) RuleBasedIdentity.instance()).getSecurityContext().insert();

This will come in handy in our next article as we discuss Seam security rules futher. Now that we have defined our authorization rule, we can apply it and perform a permission check. For this example, we want to hide the link for the deleteProfile action from our view layer. Normally, however, we would also apply this permission check in the action itself to prevent users from circumventing our view-layer check. To perform a permission check on our link, we simply add the following to our JSF file:


<s:link value="Delete Profile" action="#{member.deleteProfile(...)}"
rendered="#{s:hasPermission('member', 'deleteProfile', null)}" />

Now, when our view is displayed, the permission check will be performed prior to rendering the link. Since our admin users have been granted all permissions, the link will be rendered. If we wrote an additional rule restricting access to the deleteProfile(), our admins would still be able to view the link, while other users that fail the permission check will not see the link.

In my next article, I will discuss how to implement our two remaining authorization rules, as well as the other ways of performing a permission check within your code.

Building Seam Apps for Tomcat with JBoss Tools

2008-07-26T12:09:00.059-05:00

One of the things I really like about the Seam framework is how much easier J2EE development is, particularly with respect to pulling the various components together. With Seam, it's very easy to build applications that utilize JSF, EJBs, and JPA. However, it's often preferable to develop more lightweight applications. While EJB3 has made enterprise development much more lightweight than in the past, there is still considerable overhead associated with it, particularly when you make use of a full-blown J2EE application server like JBoss. Sometimes, it's better to utilize a simpler approach and deploy your application using a Servlet Container like Tomcat.

The JBoss Tools project goes a long way towards making the actual development process with Seam much more user friendly. JBoss Tools provides a set of Eclipse plugins designed to work seamlessly (pun intended) with the framework. One of the drawbacks of JBoss Tools, however, is its lack of proper integration when creating a project you wish to deploy to Tomcat. In this post, I'll show you the changes you'll need to make to your Eclipse projects if you want to deploy your application to Tomcat.

To start, I'll assume that you have already downloaded Tomcat and defined a Server for it in Eclipse. It's very important to remember that you need Tomcat 6, in order to have support for the Servlet 2.5 specification. Once you've downloaded and setup Tomcat in Eclipse, you're ready to create your JBoss Tools project.

Create your JBoss Tools Seam project, just as you would a regular Seam project, by clicking File->New->Seam Web Project. This will start the new project wizard. However, when selecting your Target Runtime, select your Apache Tomcat 6.0 runtime, as shown in the image below.

Click next, and select whatever project facets you may need. Click next through the wizard and fill in the appropriate fields according to your project needs, until you get to the following screen.

In this screen, be sure to select a WAR file for your deployment, as Tomcat only understands WARs, not EARs. Click Finish, and your project will be created. Now, we need to make a couple of changes to support Tomcat in our project. We'll start by ensuring that the necessary Seam libraries are a part of your project. The following libraries need to be placed in your WEB-INF/lib directory:

antlr.jar
asm.jar
cglib.jar
commons-beanutils.jar
common-collections.jar
commons-digester.jar
commons-lang.jar
commons-logging.jar
dom4j.jar
hibernate-annotations.jar
hibernate-commons-annotations.jar
hibernate-entitymanager.jar
hibernate.jar
hibernate-validator.jar
javassist.jar
jboss-archive-browsing.jar
jboss-el.jar
jboss-seam-debug.jar
jboss-seam.jar
jboss-seam-ui.jar
jsf-api.jar
jsf-facelets.jar
jsf-impl.jar
jstl.jar
jta.jar
persistence-api.jar
richfaces-api.jar
richfaces-impl.jar
richfaces-ui.jar

Once these libraries are in place, your project will build successfully and deploy correctly to Tomcat. Now, we need to look at the Resource configuration to support JPA for object persistence. First, we need to create a context.xml file and place it your META-INF directory. This is the file that Tomcat uses when deploying your project to setup the context root, as well as any resources that need to be made available to your project. With a JBoss deployment, your database settings are placed in a separate file, like MyProject-ds.xml. With Tomcat, we will place all of these settings in our context.xml file. Once completed, it will look like this:


<?xml version="1.0" encoding="UTF-8"?>
<Context crossContext="true" debug="5" docBase="myproject" path="/myproject" reloadable="true">
  <Resource auth="Container" driverClassName="com.mysql.jdbc.Driver" maxActive="20"
    maxIdle="10" maxWait="-1" name="jdbc/myproject type="javax.sql.DataSource"
    url="jdbc:mysql://localhost:3306/myproject"
    username="myuser" password="mypassword" />
</Context>

We also need to modify the persistence.xml file used by JPA. The JNDI name that Tomcat utilizes differs from what JBoss uses. Instead of "java:/", our datasource needs to start with "java:comp/env". Our persistence.xml file will now look like this:


<persistence xmlns="http://java.sun.com/xml/ns/persistence" xsi="http://www.w3.org/2001/XMLSchema-instance" schemalocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="1.0">
  <persistence-unit name="MyProject" type="RESOURCE_LOCAL">
     <provider>org.hibernate.ejb.HibernatePersistence</provider>
     <jta-data-source>java:comp/env/jdbc/MyProjectDatasource</jta-data-source>
     <properties>
         <property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5InnoDBDialect" />
         <property name="hibernate.hbm2ddl.auto" value="update" />
         <property name="hibernate.show_sql" value="true" />
         <property name="hibernate.format_sql" value="true" />
         <property name="hibernate.transaction.manager_lookup_class" value="org.hibernate.transaction.JBossTransactionManagerLookup" />
     </properties>
 </persistence-unit>
</persistence>

Lastly, we need to modify some settings in our components.xml file. First, remove the jndi-pattern="@jndiPattern@" attribute from the following line:


<core:init debug="true" pattern="@jndiPattern@">

At this point, we should be ready to deploy our applicaion and test it out. Take a good look at the log files in case your application doesn't deploy correctly. However, if you followed the above steps, everything should work as expected.

Dogtag -- Open Source PKI from Red Hat

2008-03-28T13:29:00.002-05:00

Earlier this month, Red Hat announced they were open-sourcing the PKI platform they purchased from Netscape in 2004. The resulting project, Dogtag, contains an open-source version of the code used to power Red Hat Certificate System, previously known as Netscape Certificate Management Server. While Red Hat is most known for their Linux distribution (and, more recently, their JBoss middleware products), RHCS is the basis for the world's largest PKI, developed and maintained by the U.S. Department of Defense. This PKI is responsible for nearly 10 million certificates, including those issued on the DoD's Common Access Card (CAC).

So what does this mean for the open-source community at-large? Plenty. Up until now, there has been no enterprise-class, open-source PKI solution (despite the efforts of your's truly and the Odyssi PKI project). While the community often argues that PKI is too bloated and unnecessary when compared with PGP or other lightweight products, it serves as the backbone for security in many large-scale enterprises. Being able to deploy an open-source PKI is yet one more component that Red Hat is able to provide in the application stack. This, combined with their support of the FreeIPA project for providing identity, audit, and policy management, means that Red Hat is becoming an even more formidable player in the enterprise space.

I have worked extensively with Netscape CMS, the basis for RHCS. The features, scalability, and security it provides are top-notch. In fact, many of the features in CMS served as the inspiration for things I had planned for Odyssi PKI. Will I continue development of Odyssi PKI now that Dogtag is available? Maybe, maybe not. Life seems to be getting in the way of any development projects for right now. However, in the future I may have time to continue with Odyssi PKI, or even contribute to Dogtag. We'll just have to wait and see.

Intro to JBoss Seam Security, Part 2 - Advanced Authentication

2008-02-07T11:26:00.002-05:00

In my last post, I gave a brief introduction into how JBoss Seam handles user authentication for web applications. By simply creating an authenticator class, it is possible to handle login/logout events for your application. Seam's ability to inject an identity object means that a user's identity can be accessed wherever necessary.

This method works great for simple use cases. But, what if you need more advanced functionality? The ability to enable/disable an account; the ability to manage account creation; the ability to add or remove a role programmatically. Suddenly, there is a lot more code necessary for our application to function appropriately. Thankfully, Seam has an answer to this problem.

The IdentityStore interface defines the methods for creating accounts, changing passwords, and many other account lifecycle functions. By implementing this interface and its corresponding methods, it is possible to hook into existing authentication stores, such as LDAP, and still be able to manage the account via an administrative interface.

Seam also provides a sample implementation that can be used to store accounts and roles in a database, JpaIdentityStore. The JpaIdentityStore class makes use of the Java Persistence API for storing account and role objects in a SQL database. In order to make use of the IdentityStore implementation, we need to tell Seam that we will be using it. In addition, we need to define the class that will be used as our account model object. To do this, we need to add the following line to our components.xml file:

<identity-management:jpa-identity-store name="identityStore" account-class="com.myapp.UserAccountModel"/>

As you can see, we have defined our account model as com.myapp.UserAccountModel. All that is needed is to implement this object and add the appropriate annotations so that JpaIdentityStore is able to make use of our data model.

The IdentityStore interface is simple enough that it can be implemented in a matter of hours. Whether you want to use JNDI, Kerberos, or a custom authentication method, it is simple to get up and running quickly. For more information on the IdentityStore and identity management APIs provided by Seam, take a look at this section in the documentation.

Next time, we'll take a further look at JpaIdentityStore and how to setup and configure it to work with your existing SQL table schema.

Intro to JBoss Seam Security, Part 1 - Authentication

2008-01-23T11:59:00.002-05:00

Recently, I mentioned how I had just started working with the JBoss Seam application framework. The more I have worked with it, the more impressed I have become at how it makes things that are typically difficult using standard Java EE much simpler and more streamlined. One of the areas in which Seam really shines is security. While Java EE defines JAAS for use in securing applications, it is left up to the developer to ingrain this security down to each facet of the application. With Seam, it is easy to define security constraints at all levels of an application, simply through using annotations. In addition, the complexity of authenticating users with JAAS is reduced through Seam's authenticator mechanism. This article will give an introduction to Seam authentication, and show how to write your own custom authenticator.

Seam's authenticator construct hides the complexity of managing a JAAS configuration, and allows you to implement authentication how you see fit. Perhaps your organization relies on a simple username/password combination for authenticating user accounts in LDAP. Maybe you use a SecureID token, and the accounts are stored in a SQL database. By writing your own authenticator class, or making use of publicly available ones, you can control the way authentication is done in your organization.

To get started with your own authenticator, you must first declare it in the components.xml file. This file manages much of the configuration for Seam. To add your authenticator, you simply define the class and method that will be used for authentication. For example:

<components xmlns="http://jboss.com/products/seam/components"
     xmlns:core="http://jboss.com/products/seam/core"
     xmlns:security="http://jboss.com/products/seam/security"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation=
         "http://jboss.com/products/seam/components http://jboss.com/products/seam/components-2.0.xsd
          http://jboss.com/products/seam/security http://jboss.com/products/seam/security-2.0.xsd">         

<security:identity method="#{authenticator.authenticate}">

</components>

You'll notice the #{} syntax used here. This is JBoss' expression language pointing to a class with the instance name of authenticator, where the authenticate method will be used to login a user. Now that we have declared an authenticator to Seam, we're ready to implement it. Our example will be quite simple. If the user enters a username of admin, with a password of password, they will be authenticated successfully. In addition, we will assign them to the role of admin, so that they can perform some sort of administrative function within our application. The implementation of our authenticator would look like this:

@Name("authenticator")
public class Authenticator {

private static final String valid_user = "admin";
private static final String valid_password = "password";

public boolean authenticate() {
String username = Identity.instance().getUsername();
String password = Identity.instance().getPassword();

if((username.equals(valid_user)) && (password.equals(valid_password))) {
     return true;
}

return false;
}

}

Our example is rather trivial. However, it gives a slight glimpse into how Seam authentication works. The first thing that you should notice is the @Name annotation. This annotation prompts Seam to create a bean with the name specified in the annotation. In this case, the name is authenticator, which is how we arrive at the value specified in our components.xml file. Our authenticate method will return true if authentication was successful, and false otherwise.

So how does the authenticate method get the username and password? This is done via the Identity class. The standard Identity class that comes with Seam is quite extensive, but basically provides support for a username/password combination. It is possible to subclass Identity, however, to support whatever authentication mechanisms you may need. You could implement code to support getting a SecureID token value from a user, or a SPNEGO ticket. All that is needed to make use of the Identity subclass is to add the following annotations to your implementation:

@Name("org.jboss.seam.security.identity")
@Scope(SESSION)
@Install(precedence = APPLICATION)
@BypassInterceptors
@Startup
public class MyCustomIdentity extends Identity
{ ... }

Your custom Identity subclass is now ready for use.

Now that we have our authentication classes in place, we are ready to create our login form. This is trivial to create using Seam, particularly because of Seam's use of the JBoss expression language in forms. Our login form fragment would look like the following:


<div>
   <h:outputlabel for="name" value="Username">
   <h:inputtext id="name" value="#{identity.username}">
</div>

<div>
   <h:outputlabel for="password" value="Password">
   <h:inputsecret id="password" value="#{identity.password}">
</div>

<div>
   <h:commandbutton value="Login" action="#{identity.login}">
</div>

That's all there is to it. You are now ready to authenticate users via your own custom login form and authenticator. While this is an introduction to the simplified form of authentication in Seam, it should give you a good foundation to learn and explore on your own.

Next time, we will look at how authentication is used throughout an application, not just at the entry point.

JBoss Seam - J2EE Development Framework

2008-01-17T15:16:00.000-05:00

I know I have repeatedly said that an update to OdyssiPKI is forthcoming. But, I keep getting sidetracked by interesting technologies that I just have to check out. With this update of OdyssiPKI, I decided to make it a bit more robust, and to spend more time focusing on the overall design and engineering of the application. With that came the decision to make use of EJB's, particularly EJB3. In addition, I decided to try moving away from Struts and look into Java Server Faces (JSF) for the main user interface. Of course, all these changes mean that there is a lot more that I need to get familiar with before the next release is ready.

The primary motivation for such a radical change in the project design was to make OdyssiPKI a more robust application. Currently, the user interface is very lacking. It is a simple Struts interface, and is generally not very user-friendly. In addition, the backend components of the application are very simple. Transactions, scalability, and enhanced security were not a primary focus of the first release. The combination of EJB3 and JSF makes correcting these issues much easier. The security provided by EJB3 annotations means more fine-grained control. In addition, by utilizing EJBs for the business logic, the scalability of the application is enhanced. Further, anyone who has looked at Woodstock, RichFaces, or MyFaces, and it's spinoff projects has seen what a rich interface JSF is able to provide.

The downside to all of this change? Overhead. EJB3 and JSF add a lot of overhead to the application structure. Annotations, XML files, deployment descriptors, etc. All of this creates a pretty steep learning curve, and means that development takes even longer. It is frustrating, as a developer, to have to spend so much time writing XML files when I'd rather be writing code. This is why I was very intrigued to discover JBoss Seam. Since I was already planning to develop and deploy using JBoss, it seemed like it might be worth investigating this new application framework.

JBoss Seam is an application framework designed specifically for JavaEE applications. It integrates extremely well with JSF and EJBs, as well as other Java technologies, such as Spring and Hibernate. Seam handles dependency injection as well as dependency "outjection" of various components, making it easy to assemble application services. The result? The overhead and time-to-market for JavaEE applications is drastically reduced. Seam is as close to Rapid Application Development (RAD) as I've ever seen when it comes to Java applications. And, the code generation tools included with the Seam distribution make project development even faster.

One of the most interesting tools included with Seam is seam-gen. The seam-gen utility has the ability to generate a complete project structure for Eclipse or NetBeans, simply by running Ant and answering a few questions about the overall structure of your application. Deployment descriptors, a basic interface and project files are automatically generating using the parameters you supply. In addition, you can even generate data models from an existing SQL database. All you need to do is supply the necessary connection parameters.

If you are looking for an application framework that allows you to develop robust applications very quickly, make sure you take a look at Seam. In addition, I highly recommend downloading JBoss Tools. JBoss Tools is a group of Eclipse plugins for working with JBoss, Seam, Hibernate, and other Java technologies. Getting started with Seam in Eclipse is as simple as creating a new "Seam Plugin" and following the wizard. All of the necessary files, projects, and basic classes are created for you automatically.

I'm still experimenting and getting familiar with Seam myself, but will sure to post a tutorial in the near future. In the meantime, here are a few good articles explaining Seam a bit further.

The Year of PKI? Finally?

2007-11-14T14:23:00.001-05:00

Over on news.com, there is an article saying to Expect more PKI in 2008. Since the late 1990's, several years have been deemed the "Year of PKI". The major push by the U.S. Department of Defense to build the world's largest PKI has provided a lot of momentum and investment in the PKI space. The cost of deploying smartcards across an enterprise for use with PKI, workstation login, and physical security has been greatly reduced due to the economies of scale created by DoD PKI. Microsoft's latest PKI-related products, combined with PKI integration built into Windows Server 2003 has reduced many barriers to entry for even small organizations. However, the question still remains: Will there every be a "Year of PKI?" There are many obstacles that must first be overcome before we can definitively answer that question.

Inertia
Many organizations have grown comfortable with the username/password combination. Passwords aren't secure? Just change the password policy to lengthen the minimum requirements, add special characters, etc. They see no need to move towards client-side certificates, smartcard login, or other strong authentication mechanisms until a major change in their environment requires such an effort. The same is true for intra-server communication. In most organizations, data being transmitted between web servers, application servers, database servers, etc. goes over the network in plaintext. The additional effort involved in configuring SSL for dozens of servers is often enough to justify overlooking this important security fix.

Ease of Use
In the past, rolling out an enterprise-wide PKI was an enormous undertaking in terms of the amount of work required. If an organization wanted to use client-side certificates for authentication, root certificates had to be manually installed on each workstation. Users were enrolled in a time-consuming, manual process. Additionally, the PKI products themselves were often too difficult to administer. A skilled PKI expert also came with a hefty salary. Today, however, many PKI products have become more administrator-friendly. Microsoft's Certificate Server, included within Windows Server 2003, makes deploying client-side certificates much easier by integrating with an organization's existing Active Directory infrastructure. However, this would require an organization to use Active Directory throughout, which many places are unwilling to do.

Interoperability
By this, I do not mean on interoperability between products. There are numerous well-defined standards governing PKI that even Microsoft adheres to. I am referring to interoperability between disparate PKI setups. A PKI is a hierarchical trust system created by an organization. The root certificate serves as the basis for this trust hierarchy. The problem arises when two organizations making use of PKI want to interoperate with one another. Imagine if two PKI-enabled companies, WidgetWorld and GadgetBarn, wish to interoperate with one another. Each organization has their own root certificate. Establishing a trust between these two companies would involve exchanging root and subordinate CA certificates so that SSL and client certificates can be verified all the way up the trust chain. While there are companies such as Cybertrust or Verisign that offer CA certificate signing using a ubiquitous root, that leads us to our next, and most major obstacle.

Cost $$$
Commercial, hosted PKI solutions are typically very expensive. To have certificates that are globally trusted, however, they are necessary. It is possible to create a PKI that is internal to an organization. But, as we discussed earlier, a problem occurs if you want to extend that trust outside of your organization's boundaries. The inclusion of Microsoft Certificate Server in its Windows Server 2003 product, along with open-source PKI solutions, has greatly reduced the cost of deploying an internal PKI. However, this does not address the cost of implementing, maintaining, and monitoring the PKI. In addition, there are equipment costs associated with the servers, smartcards, and smartcard readers necessary to deploy a full-scale, enterprise-wide PKI.

So, given these issues, will there ever be a "Year of PKI?" The answer is "probably not." However, this does not mean that PKI adoption will not continue to grow within the enterprise arena. As more and more organizations realize the potential severity of a data security breach, they are increasingly looking at strong authentication solutions. The benefits to implementing a PKI begin to look very attractive when weighed against the nightmare of a major security breach. In addition, PKI adoption can enable an organization to implement additional security measures, such as encrypted file systems, 802.1x network authentication, code and e-mail digital signatures, and VPN access.

While it is clear there will be no global explosion of PKI use any time soon, the future of PKI adoption does look very bright. The number of PKI implementations will most likely continue to grow in an increasingly rapid manner. However the amount of effort and investigation required with rolling out a PKI -- as with any other security-related endeavor -- will ensure that the transition will not occur overnight.

PostgreSQL Replication with Slony-I

2007-10-05T12:19:00.002-05:00

In an earlier blog post, we looked at synchronous, master-master replication in PostgreSQL using PGCluster. PGCluster's load balanacing and replication features provide PostgreSQL with high availability features that are not included in the core distribution. While PGCluster does provide an adequate solution for many environments, it is not the only replication mechanism available for PostgreSQL. In addition, its drawbacks may be too great to be deployed in many circumstances.

In this post, we look at another replication mechanism available for PostgreSQL, Slony-I. Slony-I is an asynchronous, master-slave replication system. With PGCluster, we had the ability to load balance connections to the PostgreSQL database, knowing that data that is modified in one server will be replicated across to the other server. Additionally, its synchronous nature gave us confidence that, in the event of a failure, all completed transactions will be accounted for. With Slony-I, we run into a very different type of replication. Slony-I is a master-slave system, designed to utilize one master database, and one or more slaves. Data is replicated in a one-way fashion, from the master to the slave(s). It does not include a built-in load balancing feature like PGCluster, and there is no automatic failover from a failed master to one of the slaves. Failover must be manually configured using a third-party utility, such as heartbeat. Slony-I's asynchronous nature means that, in the event of a database failure, there may be uncommitted transactions that have not been replicated across. Slony-I performs batching of transaction replication in order to improve performance. However, if the master fails prior to a batch of transactions being replicated, those transactions are lost.

On the surface, it may appear as though Slony-I is a less-than-ideal choice for database replication. However, upon further investigation, it becomes clear that Slony-I may be an acceptable choice, depending on the needs of your organization. In our example, we have one master and one slave. Our master is on a host called pgmaster, while the slave is on a host called pgslave. The database that we wish to replicate is called repl_test_db. The database consists of a single, basic table, repl_test_tbl.

Before we can begin replicating our data, we need to examine a few of the core concepts involved in Slony-I replication. The first is the notion of a 'cluster'. A cluster is simply a collection of database nodes that are connected together. These are the databases that will eventually be replicated. A 'replication set' describes the actual data (tables, sequences, etc.) that will be replicated. So, while a cluster describes the underlying database that will support replication, a replication set describes the underlying database objects that will be replicated.

To make our replication simpler, we will first declare some environment variables that will be used throughout the commands that we will issue. We will store these environment variables in a shell script called env.sh. The contents are as follows:

#!/bin/sh
REPLICATIONUSER=postgres
CLUSTERNAME=cluster_a
MASTERDB=repltestdb
SLAVEDB=${MASTERDB}
MASTERHOST=pgmaster
SLAVEHOST=pgslave
PSQL=/usr/bin/psql
CREATEDB=/usr/bin/createdb
CREATELANG=/usr/bin/createlang
CREATEUSER=/usr/bin/createuser
PGDUMP=/usr/bin/pg_dump

export REPLICATIONUSER CLUSTERNAME MASTERDBNAME SLAVEDBNAME MASTERHOST SLAVEHOST export PSQL CREATEDB CREATELANG CREATEUSER PGDUMP

Now that our environment variables have been established, we are ready to begin setting up the necessary databases. Once again, I recommend creating a shell script that will encapsulate all the commands needed to perform our setup steps. This script will first create the repltestdb database on our master host, and then ensure that the plpgsql language is installed. This step is vital, as Slony-I requires that plpgsql be installed in order to run. Once the master database has been setup, our table will be created and populated with some simple test data. From here, we begin to setup the slave host. The slave database is created, and then a pg_dump is performed to copy the initial data from our master database into the slave database. NOTE: It is important that our master and slave databases are able to communicate with each other. This will involve modifying the pg_hba.conf file to ensure that the database permissions are set properly. For more information on how to accomplish this, see the PostgreSQL 8.2 referrence manual, or the post on PGCluster replication.

For our tests, we will use a very simple table consisting of 2 columns. In our next article, we will look at the performance differences between PGCluster and Slony-I when using larger and more complex data sets. For this example, however, our simple setup will be sufficient. The SQL script looks like this:


DROP TABLE IF EXISTS repl_test_tbl;

CREATE SEQUENCE repl_test_tbl_id_seq;
CREATE TABLE repl_test_tbl (
    id INTEGER DEFAULT nextval('repl_test_tbl_id_seq') NOT NULL,
    my_str VARCHAR(32),
    PRIMARY KEY (id)
);

INSERT INTO repl_test_tbl (my_str) VALUES ('This is my #1 string');
INSERT INTO repl_test_tbl (my_str) VALUES ('This is my #2 string');

After our databases have been created and setup, we are ready to start initializing the replication sets. The slonik utility is used to issue commands to the Slony-I replication engine. The easiest way to enter these commands is using a shell script like the one in this example. Our replication script looks like this:

#!/bin/sh

echo Setting PostgreSQL environment variables...
. ./env.sh

SQL_FILE=repl_test.sql

echo Initializing master database...
${CREATEDB} -O ${REPLICATIONUSER} -h ${MASTERHOST} ${MASTERDB}
${CREATELANG} -U ${REPLICATIONUSER} plpgsql -h ${MASTERHOST} ${MASTERDB}

echo Initializing master database data...
${PSQL} -U ${REPLICATIONUSER} ${MASTERDB} < ${SQL_FILE}

echo Initializing slave database...
${CREATEDB} -O ${REPLICATIONUSER} -h ${SLAVEHOST} ${SLAVEDB}
${CREATELANG} -U ${REPLICATIONUSER} plpgsql -h ${SLAVEHOST} ${SLAVEDB}
${PGDUMP} -s -U ${REPLICATIONUSER} -h ${MASTERHOST} ${MASTERDB} | ${PSQL} -U ${REPLICATIONUSER} -h ${SLAVEHOST} ${SLAVEDB}

echo Initializing the Slony-I cluster...

slonik <<_eof_
cluster name = ${CLUSTERNAME};
node 1 admin conninfo = 'dbname=${MASTERDB} host=${MASTERHOST} user=${REPLICATIONUSER}';
node 2 admin conninfo = 'dbname=${SLAVEDB} host=${SLAVEHOST} user=${REPLICATIONUSER}';
init cluster (id = 1, comment = 'Master Node');
create set (id = 1, origin = 1, comment = 'Test repl. table');
set add table (set id = 1, origin = 1, id = 1, full qualified name = 'public.repl_test_tbl', comment = 'Test repl. table');
set add sequence (set id = 1, origin = 1, id = 2, full qualified name = 'public.repl_test_tbl_id_seq', comment = 'Test repl. table PK');

store node (id = 2, comment = 'Slave Node');
store path (server = 1, client = 2, conninfo = 'dbname=${MASTERDB} host=${MASTERHOST} user=${REPLICATIONUSER}');
store path (server = 2, client = 1, conninfo = 'dbname=${SLAVEDB} host=${SLAVEHOST} user=${REPLICATIONUSER}');
store listen (origin = 1, provider = 1, receiver = 2);
store listen (origin = 2, provider = 2, receiver = 1);

It is important to run this script as the postgres user, so make sure you su to postgres before running it. In this script, you will notice how we add our repl_test_tbl table to the replication set. It is also important to note that the repl_test_tbl_id_seq sequence must be added to the replication set as well. Slony-I requires sequences to be explicity added to the replication set, and it also requires that every table has a primary key.

Now that our cluster and replication sets have been defined, it is time to start the replication engine and subscribe to the replication set. Subscribing to a replication set tells the database to start replicating to the defined slaves. The slon command is used to start and stop the replication engine. Once again, it is best to make use of our environment variable shell script to make things easier on us. Run the first command as postgres at a command prompt on the master host, and the second set of commands on the slave host:

Master Host:
. ./env.sh
slon $CLUSTERNAME "dbname=$MASTERDB user=$REPLICATIONUSER" &

Slave Host:
. ./env.sh
slon $CLUSTERNAME "dbname=$SLAVEDB user=$REPLICATIONUSER" &

If all was successful, we are ready to subscribe to the replication sets. The shell script for performing the subscription follows:

#!/bin/sh

. ./env.sh
echo Subscribing to replication set...

slonik <<_eof_
cluster name = ${CLUSTERNAME};
node 1 admin conninfo = 'dbname=${MASTERDB} host=${MASTERHOST} user=${REPLICATIONUSER}';
node 2 admin conninfo = 'dbname=${SLAVEDB} host=${SLAVEHOST} user=${REPLICATIONUSER}';
subscribe set (id = 1, provider = 1, receiver = 2, forward = yes);

Once again, run this script as the postgres user. We are now ready to test our data replication. First, connect to the PostgreSQL master and do a SELECT to see the data in our table prior to replication. If you perform the same SELECT on the PostgreSQL slave, you should see the same data. This is the result of the pg_dump that we performed earlier. Now it is time to see if our replication is successful. Once again, connect to the master database, and, this time, INSERT data into our replicated table. If we have configured our replication properly, you should be able to connect to the slave database and see that the newly INSERTed data has been successfully replicated to the slave.

If you followed the steps listed above, you can see our replication works as expected. But, how does this compare to PGCluster for data replication? As evidenced by the number of commands we were required to issue above to get the replication working, it is obvious that Slony-I requires more administration and has a steeper learning curve. Slony-I also requires us to manually define the replication sets and the data to be replicated. With PGCluster, our databases are replicated automatically. This is due to the fact that PGCluster uses rsync as its underlying mechanism to replicate the data. However, this reliance on rsync may prove to reduce performance, particularly with large datasets. Slony-I uses a trigger-based mechanism for replicating the data. In addition, its asynchronous nature may win out in environments that do not need the high availablity that a synchronous, multi-master solution can provide.

So, it is obvious that PGCluster wins out in terms of ease of administration. But what about performance? You'll have to wait until my next article to see how each of these replication mechanisms handle larger datasets, including PGBench.

PostgreSQL Replication with PGCluster

2007-08-27T08:00:00.005-05:00

I don't know why I am fascinated with databases. I don't work with them on a daily basis. I don't know very much about them. I don't have very much experience working with them. My experience is limited to a few simple SQL queries executed on open-source databases. However, for some reason, I have always been intrigued by the idea of large, complex databases. Perhaps it is because I see the business potential in making extremely large amounts of data available for analysis within a business. Or, perhaps it is due to the unique security challenges that exist when trying to make that data available to different groups of entities, each with their own security concerns. Regardless of the reason, I have become particularly interested in high-availability databases and database replication.

In an earlier post, I briefly mentioned the PGCluster project. PGCluster is an extension of the PostgreSQL database, designed to give it synchronous, multi-master replication. Replication is one of the areas in which PostgreSQL is lacking in comparison to other proprietary databases, such as Oracle or MS SQL Server. However, after playing around with PGCluster, I have become very impressed with its capabilities. If development continues, PGCluster could offer a sound solution to a very important problem that large enterprises will encounter if they wish to role out PostgreSQL in a high-availability situation.

To that end, this article will show the basics of setting up a synchronous, multi-master replicated instance of PGCluster. For simplicity and ease of use, I am running this tutorial using virtual instances of CentOS 5 deployed using the VMWare Player. I highly recommend you obtain this free utility, as it gives you the freedom to run virtual machine instances without having to pay too much. Once you have downloaded and installed VMWare Player, you can download a pre-built CentOS 5 instance to run in the player. I made a copy of the CentOS 5 VM in another directory so that I can run 2 separate, virtual instances of it for this tutorial. You will need to configure each VM instance to have bridged network support, utilizing DHCP. In a real-world scenario, you would use static IPs for addressing, but for the purposes of this tutorial it is not necessary. After starting the VM instances for the first time, you will need to install some compiler tools in order to build PGCluster. This includes gcc, bison, and flex, as well as their respective -devel packages. To install these packages, simply run the following from the command prompt in each VM instance:

yum -y install gcc gcc-c++ flex flex-devel bison bison-devel

Once your development environments are setup, you are ready to download and compile PGCluster. There are 2 different methods for building PGCluster. The first involves downloading a patch to the original PostgreSQL source distribution. You simply apply the patch prior to compiling PostgreSQL in order to add the PGCluster support. This method is very useful if there are other patches you wish to install prior to building PostgreSQL. The second method involves downloading the complete PGCluster distribution. This distribution includes the PostgreSQL source tree already patched with PGCluster. We will use the second, full-distribution method for simplicity. As of this writing, the latest version of PGCluster is 1.7.0rc5, and is available for download here. Download the tar/gz file and unpack it on each VM instance. Building PGCluster is as simple as running:

./configure; make; make install.

This will install PGCluster to /usr/local/pgsql, by default. There is more information about the installation process available at the PGCluster website. Now that we have successfully built PGCluster, it's time to start the configuration process.

First, some terminology. PGCluster consists of 3 main components:

Clusters
Load Balancers
Replicators

A Cluster is simply a database instance. The data in the clusters is what is replicated. A Load Balancer exists to share the query load between all the databases in the replication scheme. Lastly, the Replicator is used to replicate, or synchronize, data between all the clusters. In our tutorial, we will build a replication scheme with the following components:

Two clusters, clusterdb1 and clusterdb2
Once load balancer, pglb
Two replicators, pgrepl1 and pgrepl2

Our logical design will look like this when we are finished:

It is possible to install more than one PGCluster component (cluster, load balancer, replicator) on a single system. For our example, we will put 1 cluster on each VM, with the load balancer on VM 1, and the replicators on each VM. In practice, however, you may find you receive better performance by having each node run as a dedicated PGCluster component. When finished, the physical design will look like this:

Prior to configuring PGCluster, it is important to ensure that the hostnames for all the PGCluster components can be resolved to IP addresses. You may do this using DNS, or simply add the required entries to the /etc/hosts file. In our case, clusterdb1, pglb, and pgrepl1 all resolve to 192.168.72.128. Both clusterdb2 and pgrepl2 resolve to 192.168.72.129.

It is also important that PGCluster run as a non-privileged user. In our case, this user is postgres. You can see the steps for creating the postgres user and setting the appropriate file permissions at the PGCluster install page.

PGCluster makes use of several configuration files, each specific to the component you are installing. First, we will configure each of the clusterdb instances. In the /usr/local/pgsql/data directory, you will find 3 files that need to be modified: cluster.conf, postgresql.conf, and pg_hba.conf. The cluster.conf file defines characteristics of the database cluster, as well as the replication server that will be used. The postgresql.conf is the standard PostgreSQL configuration file. The pg_hba.conf file is used for defining PostgreSQL security and access controls. This file must be modified to trust connections originating from all other databases in the cluster. Below you will find the parameters that must be added or defined for each of these files.

cluster.conf

<replicate_server_info>
<host_name>pgrepl1</host_name>
<port>8001</port>
<recovery_port>8101</recovery_port>
</replicate_server_info>
<recovery_port>7001</recovery_port>
<rsync_path>/usr/bin/rsync</rsync_path>
<rsync_option>ssh -1</rsync_option>
<rsync_compress>yes</rsync_compress>
<pg_dump_path>/usr/local/pgsql/bin/pg_dump</pg_dump_path>
<when_stand_alone>read_only</when_stand_alone>

pg_hba.conf
host all all 192.168.72.128 255.255.255.0 trust
host all all 192.168.72.129 255.255.255.0 trust

postgresql.conf
tcpip_socket = true
port = 5432

Now that the cluster instances have been configured, we can configure the replicators. We have 2 replicators defined, pgrepl1 and pgrepl2. This allows for multi-master replication. The configuration files for each instance follow.

pgreplicate.conf for pgrepl1


<Cluster_Server_Info>
   <Host_Name>clusterdb1</Host_Name>
   <Port>5432</Port>
   <Recovery_Port>7001</Recovery_Port>
</Cluster_Server_Info>
<Cluster_Server_Info>
   <Host_Name>clusterdb2</Host_Name>
   <Port>5432</Port>
   <Recovery_Port>7001</Recovery_Port>
</Cluster_Server_Info>

<loadbalance_server_info>
   <host_name>pglb</host_name>
   <recovery_port>6001</recovery_port>
</loadbalance_server_info>
<host_name>pgrepl1</host_name>
<replication_port>8001</replication_port>
<recovery_port>8101</recovery_port>
<rlog_port>8301</rlog_port>
<response_mode>normal</response_mode>
<use_replication_log>no</use_replication_log>

pgreplicate.conf for pgrepl2


<Cluster_Server_Info>
   <Host_Name>clusterdb1</Host_Name>
   <Port>5432</Port>
   <Recovery_Port>7001</Recovery_Port>
</Cluster_Server_Info>
<Cluster_Server_Info>
   <Host_Name>clusterdb2</Host_Name>
   <Port>5432</Port>
   <Recovery_Port>7001</Recovery_Port>
</Cluster_Server_Info>

<loadbalance_server_info>
   <host_name>pglb</host_name>
   <recovery_port>6001</recovery_port>
</loadbalance_server_info>
<host_name>pgrepl2</host_name>
<replication_port>8001</replication_port>
<recovery_port>8101</recovery_port>
<rlog_port>8301</rlog_port>
<response_mode>normal</response_mode>
<use_replication_log>no</use_replication_log>

Once the necessary configuration changes have been made for the clusterdb and replicator instances, we can configure the load balancer. Our load balancer will be located on the same virtual machine as clusterdb1. The configuration file for the load balancer is located in /usr/local/pgsql/etc/pglb.conf. The configuration file follows below.

pglb.conf


<cluster_server_info>
   <host_name>clusterdb1</host_name>
   <port>5432</port>
   <max_connect>32</max_connect>
</cluster_server_info>
<cluster_server_info>
   <host_name>clusterdb2</host_name>
   <port>5432</port>
   <max_connect>32</max_connect>
</cluster_server_info>
<host_name>pglb</host_name>
<backend_socket_dir>/tmp</backend_socket_dir>
<receive_port>5433</receive_port>
<recovery_port>6001</recovery_port>
<max_cluster_num>128</max_cluster_num>
<use_connection_pooling>no</use_connection_pooling>

With our configuration out of the way, we are ready to start the services and begin working with PGCluster. The order in which you start the services is important. The order is as follows:

All replicator instances
All cluster instances
All load balancer instances

To start the replicator services, run the following command as the postgres user on each replicator VM:

/usr/local/pgsql/bin/pgreplicate -D /usr/local/pgsql/etc

To start the clusterdb services, run the following command as the postgres user on each clusterdb VM:

/usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -o "-i" start

Lastly, to start the load balancer service, run the following command as the postgres user on the load balancer VM:

/usr/local/pgsql/bin/pglb -D /usr/local/pgsql/etc

You can find more information on how to start and stop PGCluster services on this page.

With our services started, we can begin testing PGCluster. All changes made to either database, either through a direct connection or one coming in through the load balancer, should be replicated across to all the other databases. The screenshot below shows a database being created. In this case, I ran the command against the clusterdb1 database. You can see the log output for this command in the top left VM. If you look at the lower right VM, you will see the log output for the replication that takes place between the two hosts.

The log output shows our database creation and replication was successful. From here, we can create tables, views, and other SQL objects and know that they will be replicated across to all the other systems we have configured.

If you run into problems, here are some common issues that my be creating the errors you encounter:

Make sure that the hostnames used for each cluster, load balancer, and replicator can be resolved to an IP address
Your firewall may be blocking incoming connections. Ensure that each host is able to communicate on all the necessary ports
Check your pg_hba.conf file if you are getting errors about rejected connections. You must set an entry for each IP address to trust in order for the data to be replicated successfully

This was just a short introduction into data replication using PGCluster. My next experiment will be to try and configure PGCluster for use with SE-PostgreSQL for added security. But, that's a post for another day.

Palm CAC Solution -- Mobile Smartcards for the DoD

2007-08-07T10:12:00.000-05:00

Recently, Palm announced the availability of a Common Access Card solution for their Treo smartphone line. For those of you not familiar with the CAC, it is a smartcard designed for use within the U.S. Department of Defense. The CAC acts as an individuals ID card, with photos and information about the user's identity, as well as a smartcard with PKI credentials for logging into a network, encrypting e-mail, etc.

Palm's CAC solution is quite interesting, particularly with respect to how the smartphone interacts with the smartcard. Palm has developed a Bluetooth-enabled smartcard reader that also acts as a badge holder. The photo, name, and expiration information is still visible, but the smartcard chip itself makes contact with a reader. This allows your smartphone to make use of the credentials on the card when sending e-mail or accessing a network.

This is a truly innovative way to handle smartcards, and with more and more enterprises evaluating the use of PKI and strong authentication, I can see this being a viable solution for the commercial world, as well. The phone itself doesn't have any bulky externally-attached readers. This makes it possible to maintain the phone's current usability and mobility, without sacrificing the security that smartcards provide.

The possibilities are endless when discussing how this technology could be used within the enterprise. Users can login to the corporate VPN from anywhere in the world, using mobile broadband connections or even WiFi (provided you have an adapter for your phone). E-mails can be digitally signed and encrypted prior to being sent. Given the lack of security in public WiFi hotspots, this is a capability that is long overdue. In addition, being able to encrypt the data on the phone itself, or lockdown access to it without the smartcard, provides the user with confidence that a lost phone will not result in data compromise.

For more information on how this product works, see Palm's Flash demo.

Java - Implement your own Service Provider Interface

2007-07-30T13:21:00.002-05:00

As I continue development on the next release of Odyssi PKI, I've tried to apply some of the lessons I've learned regarding extensibility in object-oriented code. Version 0.1 of Odyssi PKI was relatively static, in terms of what formats and technologies were supported. The request types that the CA was able to handle were statically defined. While you could implement the X509CertificateRequest interface to define your own request format, the rest of the CA was unable to handle the request beyond a certain point. This made for a very limited, inflexible design. Want to add XKMS support? Get ready to rework a LOT of code. As a result, this lack of preparation for the future was high on my list of things to redesign for the next release.

As I sought out the best way to handle this problem, I realized that working with X.509 certificate request formats/objects is very similar to the way Java already handles certificates, encryption keys, and signature algorithms. In each case, the algorithms and formats for these objects is implemented through the use of a Service Provider Interface (SPI). In this example, the SPI is the implementation of the Java Cryptography Extension that is configured for the JVM. If your preferred algorithm is not supported, perhaps one of the other JCE implementations has it. All that is needed is to configure the JVM to recognize the JCE implementation, and ensure that it is located on your classpath.

Upon realizing this, I decided that there must be a way to take advantage of this type of design to handle X.509 certificate request objects. I needed to look no further than Java's ServiceLoader class. The ServiceLoader class is used to locate classes that implement a given SPI interface or extend an abstract SPI base class. In our example, we have several classes and interfaces that will be used to handle X.509 certificate request objects. The first interface is X509CertificateRequest. This interface defines several methods that are common to all types of certificate requests: getPublicKey(), getSignatureAlgorithm(), etc. We also define a class called X509CertificateRequestFactory. This class is a factory class that creates X509CertificateRequest objects. It is also the class that will make use of the ServiceLoader object. Lastly, we need to define an interface called X509CertificateRequestFactorySpi. This interface defines the methods that must be present in all of our SPI implementations. We'll look at X509CertificateRequestFactory first, as it is the most important.

The X509CertificateRequestFactory class has a static method called getInstance() that takes as its parameter a certificate request format. This format could be PKCS #10, XKMS, CRMF, or any other format that we like, so long as we have an SPI implementation for it. The constructor for this class is private, and takes a X509CertificateRequestFactorySpi object as its only parameter. This is done because the SPI implementation object will be used by the factory class to provide all of its underlying functionality.

Now that we've discussed the general structure of the factory class, let's look at the SPI interface and the methods it defines. Since the X509CertificateRequestFactorySpi class is used to provide all of the underlying functionality for the request factory, it must provide methods such as getCertificateRequest(...) that will be used to generate the request object itself from another object (byte[], String, XML document, etc.) In addition, another method isSupported() is defined, which takes as its only parameter a String, denoting the request format. This method returns true if the request format is supported by this SPI implementation.

So how do we use the ServiceLoader class? The ServiceLoader class looks for a provider configuration file located in the META-INF/services directory of the SPI implementation's JAR file. This file is the fully-qualified binary name of the services type. That is, if our request factory's full name is net.odyssi.certserv.x509.request.X509CertificateRequestFactory, the provider configuration file would be called net.odyssi.certserv.x509.request.X509CertificateRequestFactory, and would be located in the META-INF/services directory of the implementation JAR file. This file contains the name(s) of the SPI implementation class(es) contained within the JAR. So, for example, if we wanted to support PKCS #10 certificate requests, our provider configuration file may have one entry that reads:

net.odyssi.certserv.x509.request.impl.PKCS10CertificateRequestFactory

The PKCS10CertificateRequestFactory class provides all of the dirty work in generating an X509CertificateRequest object from an arbitrary source, such as an InputStream or a byte[]. Now that we've defined our provider configuration file with our SPI implementation class, we're ready to put the ServiceLoader class to work.

Within the getInstance() method of the X509CertificateRequestFactory class, we would have the following:


public static X509CertificateRequestFactory getInstance(String format) {

ServiceLoader sl = ServiceLoader.load(X509CertificateRequestFactorySpi.class);
Iterator it = sl.iterator();
while(it.hasNext()) {
   X509CertificateRequestFactorySpi impl = (X509CertificateRequestFactorySpi)it.next();
   if(impl.isSupported(format)) {
       return new X509CertificateRequestFactory(impl);
   }
}

return null;
}

In this method, the ServiceLoader sl = ServiceLoader.class(X509CertificateRequestFactorySpi.class) line instructs the ServiceLoader to find all implementations of the X509CertificateRequestFactorySpi interface. From here, we iterate through the returned results and see if an implementation can be found for the given request format. The returned X509CertificateFactory would then used the SPI implementation to provide its underlying functionality for getCertificateRequest(...), etc.

The ServiceLoader class provides a great mechanism for adding functionality and future-proofing your applications. Unlike other mechanisms such as Spring's dependency injection, no modifications are necessary for your code to take advantage of the new SPI implementation. The SPI implementation JAR file simply needs to be located on the classpath for the functionality to be available. By making use of a well defined SPI, it is possible to interchange components with your application without losing functionality. It also provides a great way of adding new capabilities to your already existing applications. For more information about ServiceLoader, take a look at the javadocs, or this article.

PostgreSQL Replication - PGCluster

2007-07-26T09:56:00.000-05:00

Earlier, I did an article on MySQL multi-master replication. While MySQL is a very fast database, able to provide the vast majority of features that most people need, there are definite shortcomings. As a result, I've begun looking more and more into PostgreSQL as my database of choice for my development work. The major shortcoming for me, however, has always been PostgreSQL's replication and failover support. The Slony-I project provides replication, however it is asynchronous and master-slave. In many environments, this just isn't good enough. I came across PGCluster, only to see on its project site that the latest code was for PostgreSQL 7.3.

At first, I thought that this was reason enough to abandon PostgreSQL and focus primarily on MySQL. However, I have since found that I am mistaken. The PGCluster site that I initially came across was an old site for the project. The new PGCluster site shows that PGCluster has been updated for PostgreSQL 8.2. This is great news for anyone looking to use PostgreSQL in an enterprise-wide capacity. PGCluster provides synchronous, multi-master replication. It is designed for high-availability, as well as load-balancing. While I haven't had a chance to install the latest version of PGCluster, I will be giving it a long, hard look in the near future.

While working on the Odyssi PKI project's next release, I've spent a bit more time focusing on the database aspect. Certificate Authority servers are typically considered to have high-security requirements, so they are often run using dedicated database servers. You wouldn't typically want your certificate data in the same database server with your general business data. Normally, the database instance is dedicated to the CA, and is hardened for security to prevent compromise. However, in situations where high numbers of certificates are issued, performance and scalability may become a factor. In a hosted PKI environment, for example, large numbers of certificates being issued may put a strain on the database server. In addition, CA servers have high availability requirements, particularly for CRL issuance, etc. The need for failover replication becomes apparent.

With PGCluster, it is possible to implement a database architecture that allows for the performance and reliability needed for this type of scenario. The example provided on the PGCluster website shows a simple replication scenario between 3 servers; two are located in close proximity, with the third connecting remotely over a VPN. When designing a database layout for an Odyssi PKI deployment, you would typically want to have a minimum of 2 databases, acting in a load-balanced, multi-master configuration. This will provide you with failover capability in the event of a server failure, as well as the ability to split processing between servers.

Starting with Odyssi PKI's next release, I plan to include some documentation outlining recommended architecture, best-practices, etc. for designing a PKI. Now that I have discovered PGCluster is not a dead project, I will be sure to use it in my examples.

Odyssi PKI - 1,000 Downloads

2007-07-20T08:56:00.000-05:00

As of this morning, version 0.1 of Odyssi PKI has been downloaded 1,000 times. My thanks to all those who downloaded the program. I am currently working on version 0.2, which will represent a major rewrite and restructuring of the entire codebase. Version 0.1 was more intended as a proof-of-concept. In writing it, I was able to experiment with several technologies I hadn't played with before, like Spring or Hibernate. The development turned out to be a great learning experience.

Version 0.2 will be much better from a design and functionality standpoint. It is being rewritten from the ground up, and will include many functions and features that were missing in version 0.1. This includes:

Easier CA administration, through the use of a web-based admin console
Easier creation and management of Registration Authorities
Enhanced security requirements, such as enforcement of strong authentication in the Registration Authority console
CRL support
XKMS (possibly)
Enhanced certificate template support, allowing admins to define the characteristics of different types of certificates that will be generated
Support for numerous X.509 certificate extensions
Stand-alone release (using an included distribution of Apache Tomcat) for easier deployment and administration

This is just a small list of the features I plan on including in the next version. I have no firm dates on when it will be released, as I'm working on it in my free time (which is minimal, now that I have a toddler). However, work is progressing quickly and I hope to have something posted really soon.

Free at Last, Free at Last! FiOS is Here!

2007-05-30T08:40:00.000-05:00

This morning, I was awakened by the sound of jackhammers outside of our house. Normally, jackhammers at 7:00 would be quite an unpleasant situation. However, this morning, that sound is a welcome one. Why? Because it means that, soon, there will be a fiber optic cable buried in the ground on our street. And it is this cable that represents my freedom. Freedom from our local cable company, Comcast.

Anyone who has ever had Comcast for their cable or Internet service can most likely relate to my feelings about the company. Slow service. Outrageous prices (and frequent price increases). Terrible customer support. The list goes on, and on. Soon, however, I will be able to break free of the shackles of oppression (Comcast), and experience the freedom that is Verizon FiOS. Now, don't get me wrong. I'm well aware of the fact that FiOS will likely cost me just as much as Comcast does right now. However, the mere fact that I'm able to get away from Comcast in the first place makes it all worth it.

When I purchased my house (about 6 years ago), my bill for cable and Internet service was about $80. This included basic cable, and a 4Mbps (advertised) Internet connection. At the time, I had no complaints whatsoever. My cable rarely, if ever, went out. And my Internet connection was always speedy and reliable. Over time, however, the situation has changed dramatically. My combined bill is now over $100. And what has the addition $20 gotten me in 6 years? 2 additional TV channels (only one of which I'm remotely interested in), and an increased in the advertised Internet connection speed. Note the word advertised. The advertisement in question is quick to point out that this is a maximum connection speed. In other words, Comcast could provide me with all of 1bps, and would still be offering the service they advertise.

In reality, my Internet service has gotten consistently worse over the last 6 years. In the first 3 years that I lived there, I lost service only twice. In both instances, I was able to resolve my problem by simply disconnecting my cable modem, waiting a couple of minutes, and reconnecting it. Now, however, I am faced with connection problems on an almost weekly basis. And, sadly, most of these are issues I am unable to resolve. Periodically, I will end up with little to no connectivity. The indicator light on the modem tells me that there is a signal coming in. But, for whatever reason, I can't connect to anything. Other times, I am faced with DNS resolution issues. I can get out if I know the IP address. But, good luck relying on Comcast's DNS servers. What upsets me most, though, is that my downstream service has gotten progressively slower over the last 3 years. My bill has gone up, my connection speed has gone down.

I'm sure that many of these problems boil down to one thing: Comcast doesn't feel compelled to spend money on improving service when they've already got a monopolistic hold on a market. Why invest in infrastructure improvements if your customers don't have an alternative? Such sentiment exists whenever a company has a monopoly. However, when an attractive alternative opens up, customers who feel abused by a monopoly are often quick to migrate to that alternative. Look at Microsoft, and the monopoly it has on the software (particularly Operating System) market. Consumers, fed up with the ongoing instability and security issues found in Windows, have begun migrating to Apple's Mac OS X. While Apple's market share is still small compared to Microsoft, it is growing steadily, particularly in the notebook computer segment. When I walk through our neighborhood, I inevitably hear someone talking to one of the workers laying Verizon's fiber optic cable. The discussion always seems to start out the same way: "When can I call and have FiOS installed at our house?" Consumers are chomping at the bit to find a better alternative.

While Comcast still has a stranglehold on numerous markets (including ours, for the time being), this situation demonstrates how quickly things can change in any market. Dell at one point had a commanding lead over its competitors, based primarily on its reputation for quality and good customer service. However, recent declines in quality combined with poor customer support due to outsourcing has caused Dell's market share to slip, and its reputation to be tarnished. Ed Catmull, one of the creative geniuses behind Pixar, once said, "Quality is the best business plan." By offering its customers sub-par products at an ever-increasing price, Comcast is risking losing those customers to competitors such as Verizon or DirecTV. I, for one, will be scheduling an installation appointment just as soon as the Verizon trucks pull out of our street.

New Odyssi PKI Project Location

2007-05-11T09:55:00.000-05:00

I have modified the Sourceforge project location for OdyssiCS. The project has moved to its new page, http://www.sourceforge.net/projects/odyssipki , and has been given a new project name, Odyssi PKI, to reflect the overall goals of the project. With the inclusion of some additional features, such as an OCSP responder and GUI-based certificate/key management tools, it became clear that I was developing more than just a Certificate Authority server. As a result, I've renamed the project Odyssi PKI to reflect the suite of tools that will eventually be released. Although the project page has changed locations, the website for the project remains the same.

OdyssiCS Update and Milestones

2007-05-04T13:44:00.000-05:00

This morning, I logged into the SourceForge project page for OdyssiCS. I was curious to see how many downloads of the application have occurred. To my surprise, the number of downloads since release 0.1 last year has topped 800. I don't know how many of these downloads resulted in any substantial use, but it's still refreshing to know that people are at least looking at it. To all of you that have downloaded and tried OdyssiCS, thank you. And, please, give me your feedback! It will go a long way to helping me in my development.

I have been working on version 0.2 in what little spare time I have. I've been fortunate to learn a great deal about Hibernate, Spring, and general Java security since the first release. The main areas I have been focusing on for release 0.2 are:

Security
Ease of Administration
Inclusion of more advanced features

Release 0.1 was rather weak in terms of security. While I did perform basic group membership checks, I realized that I was not designing for security as much as I'd originally intended. As a result, I have gutted a great deal of the original code to ensure that it is designed with security in mind from the ground up. I've also been spending a bit of time working on a secure JAAS-based application framework that I hope to have included in version 0.3.

Administration in release 0.1 was also quite lacking. In an effort to get the first release out there, I didn't spend as much time on developing an administrative interface. This includes both CA server administration, as well as certificate administration for Registration Authorities. I've sketched out quite a few ideas that I would like to incorporate into the web interface for administration. Hopefully, this will make OdyssiCS easier to deploy and maintain, making it viable for some real-world work. Additionally, for version 0.3 I plan to include an embedded Tomcat server and Windows GUI installer to make installation and configuration even easier. My plans for version 0.3 are still quite sketchy, but hopefully it will represent a really high-quality, feature-rich release.

Lastly, there was quite a bit of functionality missing in version 0.1. This is being addressed in version 0.2, starting with the use of certificate extensions and templates. Part of the power of X.509 certificates is the ability to include certificate extensions. These extensions outline additional characteristics about the certificate, such as the constraints that exist pertaining to what it can be used for, CRL information, and policies that apply to the certificate. I had some code in version 0.1 for working with extensions, but just wasn't happy with how it turned out. I shelved it, deciding it would be best for the next release. Certificate templates provide a way to define the characteristics of the different types of certificates a CA can create. For example, SSL server certificates might have different properties (extensions, validity periods, key sizes, etc.) from e-mail certificates. This will add a great deal of functionality to OdyssiCS, and I look forward to having that code completed.

Some of the features I plan on implementing for version 0.2 include:

Certificate revocation, with full CRL support
OCSP (Online Certificate Status Protocol) for reporting certificate revocation information
Completely redesigned GUI for both end-entities and administrators
Enhanced security features throughout all domains of the application
Revised SQL schema, and SQL scripts for MySQL, PostgreSQL, and other SQL databases
X.509 certificate extensions and certificate templates

This is just a sample of some of the things I'm working on for the next release. I don't have a timeline for it, as my free time for development has been limited. Please feel free to leave a comment if there are additional features you would like me to investigate for this release. I've already started pondering what I want for version 0.3 (XKMS, JAAS-based security framework, JMX management, etc.) so please pass on your suggestions!

Hibernate CompositeUserType and X509Certificate

2007-04-02T10:54:00.005-05:00

Often, when working with Hibernate as your ORM tool, you find the need to persist an object to the database that doesn't match one of the supported types. For example, what if you wanted to persist an XML object? One such way would be to convert the XML to a String type and persist it that way. Of course, that messes up our object model, doesn't it? Wouldn't it be nice if a setXXX() method could handle the XML object directly? Using Hibernate's UserType object, we have the flexibility to handle these types of conversions outside of our object model.

I was faced with a very similar problem while reworking the way X.509 certificates are persisted in OdyssiCS. Previously, I would conver the certificate to a byte[] and store its base64 encoded representation in the database. This resulted in a very ugly object model. I contemplated making use of a UserType object for my X.509 certificate, but was then faced with a secondary problem: How would I be able to perform queries against properties of the certificate? What if I wanted to locate a certificate with a specific serial number? One approach would be to maintain those properties separately. However, this results, again, in a polluted object model. After further review, I decided upon Hibernate's CompositeUserType object. A CompositeUserType is an extension of UserType that allows for properties to be derefferenced when performing a Hibernate query. Since I couldn't find much information about how to work with CompositeUserType objects, I decided to post some of my findings here to serve as a reference to others. First, let's look at what we want our end result to be.

In my object model, I have a CertificateModelImpl class that contains one method related to what we're doing with CompositeUserType: getCertificate(). This method returns the X.509 certificate object. Our database has a table called certificate_tbl with the following columns:

ID -- A simple incremented row identifier
SUBJECT_ID -- Corresponds to the table containing our subject distinguished names
START_DATE -- The start of the certificate validity
END_DATE -- The expiration date for the certificate
SERIAL_NUMBER -- The certificate serial number
CERTIFICATE_DATA -- Contains the certificate in byte[] format

In our relational model, SUBJECT_ID is a foreign key pointing to another table. For our discussion, it is not relavant. We would like to focus on the remaining columns. Our end goal is this: We want to be able to call setCertificate() on CertificateModel, passing an X509Certificate object as a parameter. We also need to be able to use properties of the certificate, such as serial number and expiration date, in a Hibernate query. We want to store the certificate properties in the corresponding columns of the table, but don't want them available to our object model. If these properties were available, we could run into an issue with data inconsistancy.

To start implementing our X509CertificateUserType object, we must implement a couple of preliminary methods. The returnedClass() method tells Hibernate what type of object we are dealing with. In our case, we return an X509Certificate.class object. The equals() and hashCode() methods are pretty standard fare, acting as they normally do in Java. There are a couple of other methods that must be implemented, such as deepCopy() and isMutable(). Take a look at the Hibernate Javadocs for more information on what the remaining methods accomplish.

Our first goal is to be able to store an X509Certificate object in the database as a byte[] and retrieve it later as an X509Certificate object. The two methods of CompositeUserType that accomplish this are nullSafeSet() and nullSafeGet(). The nullSafeSet() method is responsible for converting the certificate to a byte[] and storing it in the appropriate column. This is also where we store the other certificate properties, such as expiration information and serial number. For our puirposes, this is what the implementation of this method looks like:


public void nullSafeSet(PreparedStatement preparedStatement, Object object,
    int i, SessionImplementor sessionImplementor) throws HibernateException, SQLException {
  X509Certificate cert = (X509Certificate) object;
  Hibernate.DATE.nullSafeSet(preparedStatement, cert.getNotBefore(), i);
  Hibernate.DATE.nullSafeSet(preparedStatement, cert.getNotAfter(), i + 1);
  Hibernate.BIG_INTEGER.nullSafeSet(preparedStatement, cert.getSerialNumber(), i + 2);

  try {
    preparedStatement.setBytes(i + 3, cert.getEncoded());
  } catch (CertificateEncodingException e) {
    throw new HibernateException(e);
  }
}

You see here that we, first, set the additional properties for certificate. How do we know which columns the values are set in? This is set as part of our Hibernate mapping file, which I will explain later. From there, we convert the certificate to a byte[] and set that in the database as well. We're now ready to implement the method for retrieving the certificate from the database. Our nullSafeGet() implementation looks like this:



public Object nullSafeGet(ResultSet resultSet, String[] strings, 
          SessionImplementor sessionImplementor, Object object)
          throws HibernateException, SQLException {
   X509Certificate cert = null;
   byte[] certData = resultSet.getBytes(strings[3]);
   if (!resultSet.wasNull()) {
      try {
         CertificateFactory cf = CertificateFactory.getInstance("X.509");
         cert = (X509Certificate) cf.generateCertificate(new ByteArrayInputStream(certData));
      } catch (CertificateException e) {
         throw new HibernateException(e);
      }
   } 
   return cert;
}

Here, you'll se we create a CertificateFactory object to parse our byte[] into an X509Certificate object. At no point do we concern ourselves with the additional certificate properties we set earlier. Our only concern is the certificate itself. At this point, we can store and retrieve an X509Certificate object with no problems.

We now need the ability to reference properties of an X509Certificate in a Hibernate query to simplify with searching. The getPropertyNames(), getPropertyTypes(), getPropertyValue(), and setPropertyValue() methods of CompositeUserType assist with this goal. The getPropertyNames() method returns the names of the properties we wish to make available when querying for a certificate. In our case, these values are:

issueDate
expirationDate
serialNumber
certificateData

The getPropertyTypes() method specifies the Hibernate type that corresponds to each of these properties. So, in our case, the following types would be returned:

Hibernate.DATE
Hibernate.DATE
Hibernate.BIG_INTEGER
Hibernate.BLOB

The getPropertyValue() method is responsible for determining which property is being queried and then returning the appropriate value. In our case, the implementation looks like this:


public Object getPropertyValue(Object object, int i) throws HibernateException {
   if (object == null) {
      return null;
   }
   X509Certificate cert = (X509Certificate) object; 
   if (i == 0) {
      return cert.getNotBefore();
   } else if (i == 1) {
      return cert.getNotAfter();
   } else if (i == 2) {
      return cert.getSerialNumber();
   } else if (i == 3) {
      try {
         return cert.getEncoded();
      } catch (CertificateEncodingException e) {
         throw new HibernateException(e);
      }
   } else {
      return null;
   }
}

The final method, setPropertyValue(), does nothing in our implementation. We, obviously, don't want the certificate properties to be modifiable, so our method implementation simply does nothing and then returns.

Now that we have implemented the necessary methods, we need to create the Hibernate mapping for our object. In our CertificateModelImpl.hbm.xml file, we have the following mapping for our certificate property:


<property name="certificate" type="net.odyssi.certserv.dao.hibernate.model.X509CertificateUserType">
   <column null="true" type="date" name="start_date">
   <column null="true" type="time" name="expiration_date">
   <column null="true" type="integer" name="serial_number">
   <column null="true" type="blob" name="certificate_data">
</property>

That's all there is to it! Hibernate can now store and retrieve X509Certificate objects with no problem, and queries can reference properties of that certificate. To see the finished product, take a look at X509CertificateUserType in the OdyssiCS CVS repository.

Security-Enhanced PostgreSQL

2007-03-30T10:49:00.000-05:00

Recently, I came across a posting announcing the release of a SELinux-enhanced version of PostgreSQL. SE-PostgreSQL makes use of the labeling and Mandatory Access Control (MAC) features that SELinux provides. This is a tremendous addition to enterprise security, particularly in the government and financial fields.

What SE-PostgreSQL provides is a mechanism for controlling access to information in a database in a very fine-graned manner. The example provided in the announcement demonstrates this using a table containing beverage information. In this example, the table contains several columns related to different types of beverages (name, price, type of beverage, quantity on hand, etc.). The table is then altered to require a SE-Linux security context to access specific parts of this data. For example, one security context is required to access the table at all. An additional, higher-security context is required to access rows where the beverage is not a soft-drink. If you look at the output provided in the sample queries, you will see that these rules are applied based on the user's security context. The amount of customization doesn't end there, however. You can apply access control based on rows, columns, tables, databases, etc. While this level of customization may be intimidating to smaller enterprises, larger enterprises will welcome the flexibility it gives them.

This type of capability has huge ramifications for the use of PostgreSQL, Linux, and open-source in general within the government and financial sectors. Data of different government classification levels can now reside in the same database. A unified database can be used in financial institutions with assurance that the information cannot be modified by unprivileged or unauthorized users. My small amount of SQL experience has largely revolved around open-source databases, such as MySQL and PostgreSQL. My day job does not require much SQL work, so as a result I only play around with it on my free time. While MySQL seems to be the most widely used, my personal preference is for PostgreSQL. True, it lacks some of the features MySQL has, such as clustering. However, it more than makes up for it in other ways, such as its strict transaction handling abilities. I've always felt PostgreSQL's security architecture was superior as well, with its ability to use external authentication sources (Kerberos, LDAP, etc.) for users. The addition of SELinux support to this mix has put PostgreSQL in a class unmatched by any other open-source database (and many closed-source ones as well).

CentOS 5-beta -- First Impressions

2007-03-15T10:51:00.000-05:00

Last night, I installed the beta release of CentOS 5 to take it for a test drive. For those of you looking for screenshots, there are several screenshots of Red Hat Enterprise Linux (RHEL) 5 available here. Although the artwork is different in CentOS, everything else is the same. These screenshots should give you a general idea of what the experience is like. While I haven't had a chance to play with all of the new and enhanced features, I've been quite impressed with what I have seen so far. I started by installing the beta on my HP Pavilion zt3000 laptop. My biggest criticism of those who claim that "Linux is ready for the desktop" has been issues with running it on a laptop. From ACPI support, poor hardware support, and an inability to function the way a laptop should, I have been less than thrilled with several Linux distributions.

For starters, I was pleasantly surprised with the updates to Anaconda, the system installer. The entire install process has been streamlined, requiring far less user intervention, even for more advanced configurations. One of the things I like most about Windows XP installations is that, for the most part, it is "set it, and forget it". You answer a few questions about networking and such, and the rest of the install is automated. CentOS 5 follows this philosophy with the updated installer.

One thing that was obvious from the first time I booted into CentOS 5, is that it is FAST. Really fast. In CentOS 4.4, it took approximately 10 seconds from login to GNOME desktop. With CentOS 5, this time has been reduced to 2 seconds. While this is a pretty trivial achievement, it goes a long way to improving the usability of the system. Speaking of usability, CentOS won me over during the boot process. Why? Because X Windows, by default, was set to my laptop's optimum resolution: 1680x1050. In every other Linux distribution I have used, it defaulted to some other non-widescreen resolution, and I was forced to manually change the configuration file. Not any more. In addition, when I later plugged the laptop into its docking station to use my external 19" LCD, it immediately changed its resolution to the appropriate 1280x1024. This is such a long overdue change in system behavior.

Once I was logged in, I began taking a quick tour of the system. The new Clearlooks theme looks great, and the crispness of screen fonts is incredible. I've never seen a Linux distribution that displays fonts this clearly. This release also includes Compiz, the compositing engine for X Windows. Compiz provides all the nifty (yet pointless) visual effects for the desktop, similar to what Mac OS X is capable of. This was one of the things I was most curious about, as I have not had a chance to work with a Compiz-enabled system. My impressions? Not bad. The wobbly windows are definitely a neat trick, but I don't like that it takes a split second for the windows to snap into place and regain their focus. One thing I didn't get a chance to investigate was the load on the CPU with Compiz enabled. Compiz is OpenGL accelerated, and I wanted to make sure that the extra load was forced off to the graphics card instead of the CPU. I'll take a look at this tonight.

This evening, I plan to investigate some of the server and developer-oriented features of CentOS 5. Since I spend most of my time in Linux doing Java development work, I'm curious as to the performance of Eclipse, Intellij Idea, MySQL/PostgreSQL, and Tomcat. If the server performance has increased in the same way as desktop performance, I think I will find myself spending a lot more time in Linux, and a lot less time using Windows XP.

Smartcards at Disneyland?

2007-03-14T10:52:00.000-05:00

I stumbled upon this blog entry, showing a picture of the new hotel room keys in use at Disneyland Paris. You're not mistaken, they look like smartcards. It seems every year in recent memory has been called, "The Year of the Smartcard" or, "The Year of PKI". I can certainly imagine some innovative uses for these smartcards. Since they can be used for payments at all of the restaurants and stores within the park, what about a Kerberos-enabled payment system? Or, load the cards with credentials enabling a user to download photos from their vacation when they get home (so long as they have a reader attached to their PC). While I'm not sure what the backend technology behind the cards does, it's not the first time that Disney has used cutting-edge security technologies at their parks and resorts. In 2005, Disney implemented biometric finger scans for all guests at Walt Disney World. However, they used similar technology for annual passholders even before that.

Oracle Linux Download

2006-10-26T11:00:00.000-05:00

Yesterday, Oracle announced that they will be supporting Red Hat Linux, as well as releasing their own clone of the OS, just like CentOS and others have done. There had been speculation for weeks that Oracle was going to get into the Linux business, either by creating their own distribution, or purchasing a Linux company. While there have been some negative reactions to this announcement, I view it as a great opportunity, and a means of forcing more innovation into the market. The download itself is available here. You simply need to fill out a form to gain access to the files.

Those of you who have read previous entries in my blog may realize that I'm a fan of the Red Hat clone, CentOS. I've used it on laptops, desktops, and servers, and have had very few problems. The stability it provides is far better than I've experienced with Fedora, SuSE, or other distributions. However, if I wanted to install it in a corporate environment, my support options would be limited. Red Hat charges a great deal for support, even rivaling Microsoft in terms of pricing. With Oracle now offering support, the cost of deployment is greatly reduced. Some have argued that this is a low-blow by Oracle, as they are simply repackaging Red Hat's work and distributing it as their own. That is exactly what they're doing. However, that is the risk with open source software. Just because Red Hat is "loved" by the open source community doesn't mean it should be treated any differently.

For the past several years, open source advocates have argued that it is a superior business model. The thought that releasing a product as free, open-source software and charging support will give you a lock on a given market is laughable, at best. A company like Red Hat cannot compete with Oracle in terms of size and resources. As a result, they are vulnerable to having their prices undercut by a company that is able to deliver better support for less money. MySQL, Alfresco, and all other open-source companies are vulnerable to this as well. The key is not just providing support, but providing some additional value that your competitors are incapable of providing. And, currently, Red Hat is not taking these measures.

Red Hat provides no value-added resources on top of their existing OS distribution. For those individuals who don't need support, there is no incentive to purchase a commercial distribution of the product. They can simply download CentOS or one of the other clones that are available. Now that Oracle is offering cheaper support, there is even less of an incentive for a company to purchase support from Red Hat. If, however, Red Hat began to provide offerings that only they were capable of, they would regain their advantage in this area. Oracle has learned this lesson the hard way. After growing tired of being "just a database company", they began developing and acquiring applications that were designed to run on top of their database as a complete stack. By providing an all-in-one solution, companies were able to get all of their products in one place, from one vendor, with one support contract. Red Hat does have some closed-source applications in their product library, such as their Certificate System which was purchased from Netscape some time ago. However, this is such a niche product that it is not important to the majority of Red Hat's customers. Providing other applications, however, may provide them with an advantage. For example, an identity management system similar to Active Directory, using their LDAP directory server and Kerberos for single sign-on. Or, tightly integrated management applications for administering Apache, LDAP, MySQL/PostgreSQL, and other system services.

I am not surprised by Red Hat's sudden drop in share price following this announcement. For existing Oracle customers, there is now a very good reason to drop support contracts currently offered by Red Hat. For those not running Oracle on Linux, they now have a very good reason to think about it.

MySQL Master-Master Replication

2006-10-09T11:01:00.000-05:00

I came across a great article detailing how to accomplish Master-Master Replication on MySQL 5. While there are other technologies that could be used to create high availability MySQL instances, such as MySQL Cluster, these often provide a less-than-ideal solution because of the additional requirements they impose (e.g. the use of in-memory databases or a different storage engine). The solution provided in the article is to use circular replication. That is, if we have 2 databases, A and B: A is a master to B, while B is a master to A. Because each node acts as both a master and a slave, changes to either database get propigated to the other. Take a look at the article to find out more.

Great Workspaces

2006-10-06T11:02:00.000-05:00

I came across a great article depicting some of the best workspaces in companies around the world. Pixar, Google, Red Bull, etc. The pictures show just how creative these offices can be, and show that this creativity can lead to creativity in their employees. As someone who has worked in a private office, cubicle, and now a shared workspace, I can certainly understand how offices like this can make the daily 9-5 grind a much more pleasant experience.

Complex Many-to-Many Relations in Hibernate

2006-09-21T10:56:00.005-05:00

SQL makes querying structured data very easily, especially when the data in tables is related. Using simple joins, it's possible to query for data in two separate tables that shares a common element. For example, imagine if we have a table users_tbl that contains our user account information, and a table roles_tbl that contains our roles information. Additionally, we make use of a join table users_roles_tbl that allows us to have many-to-many relationships between our users and roles. To create our tables, we could use the following commands (on MySQL 5):


create table users_tbl (
   id INT AUTO_INCREMENT NOT NULL,
   username VARCHAR(32) NOT NULL,
   PRIMARY KEY (id)
) TYPE=INNODB;

create table roles_tbl (
     id INT AUTO_INCREMENT NOT NULL,
     rolename VARCHAR(32) NOT NULL,
     PRIMARY KEY (id)
) TYPE=INNODB;

create table users_roles_tbl (
    user_id INT NOT NULL,
    role_id INT NOT NULL,
    FOREIGN KEY (user_id) REFERENCES users_tbl (id),
    FOREIGN KEY (role_id) REFERENCES roles_tbl (id),
    PRIMARY KEY (user_id, role_id)
) TYPE=INNODB;

Suppose we have created 3 users: user_a, user_b, and user_c. Additionally, we have created 2 roles: role_a, and role_b. Our first user, user_a, is a member of role_a. Our second user, user_b, is a member of role_b. And, our third user, user_c, is a member of both role_a and role_b. We can see these users and roles displayed with the following SQL SELECT statement:


SELECT DISTINCT u.username, r.rolename FROM users_tbl u
   INNER JOIN users_roles_tbl j ON u.id = j.user_id
   INNER JOIN roles_tbl r ON r.id = j.role_id;

There are many instances in which we would like to find members of specific role, such as for access control. If we wanted to find all members of role_a, a simple modification to the previous SELECT statement will do the trick:


SELECT DISTINCT u.username, r.rolename FROM users_tbl u
   INNER JOIN users_roles_tbl j ON u.id = j.user_id
   INNER JOIN roles_tbl r ON r.id = j.role_id WHERE r.rolename = 'role_a';

In this example, user_a and user_c are returned, since they are the only members of role_a. But what happens when we want to perform a more complex query? For example, what if we only wanted to return users who are members of role_a AND role_b? This is where things become a bit more complicated. The IN statement allows us to perform a logical ORing of values. This is what we would use if wanted to find users who are members of role_a OR role_b. But what if we want users who are members of BOTH roles?

In this case, our SQL SELECT statement becomes a bit more complicated. Our SELECT statement would now look like this:


SELECT DISTINCT u.username FROM users_tbl u
   INNER JOIN( roles_tbl r INNER JOIN users_roles_tbl j
   ON ( ( r.id = j.role_id ) AND ( ( r.rolename = 'role_a' )
   OR ( r.rolename = 'role_b' ) ) )) ON ( u.id = j.user_id )
   GROUP BY u.id HAVING COUNT( j.user_id ) = 2;

As you can see, this SELECT statement gets very complex, very quickly. However, it accomplishes exactly what we like. Running this statement provides us with only user_c, the only user who is a member of both role_a and role_b.

Our question now becomes, how do we accomplish the same thing using Hibernate? For example, we have our core domain model objects, UserModel and RoleModel which have the appropriate Hibernate mappings configured. One way we could accomplish our goal is to create a many-to-many relation in Hibernate for our UserModel object that contains all the roles a user is a member of. We could then query on role_a, query on role_b, and then determine the intersection of those two Collections that are returned. However, this is a very expensive and time-consuming operation.

There is a much better way to accomplish our goal. We can create an additional domain model object that represents the data stored in our join table, users_roles_tbl. This object can be called UserRoleRelation. For these examples, I will be using XDoclet to generate our Hibernate mapping classes. Our UserModel class would look like the following:


/**
 * @hibernate.class table="users_tbl" lazy="true"
 */
public class UserModel {

    private Serializable identifier = null;
    private String username = null;

    /**
     * @hibernate.id column="id"
    *     generator-class="native"
    *     type="java.lang.Integer"
     */
    public Serializable getIdentifier() {
        return identifier;
    }

    public void setIdentifier(Serializable identifier) {
        this.identifier = identifier;
    }

    /**
     * @hibernate.property column="username" not-null="true"
     */
    public String getUsername() {
       return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }
}

Our RoleModel object would be configured similarly, but with the Hibernate table as roles_tbl. As for our UserRoleRelation object, it would be defined as:


/**
 * @hibernate.class table="users_roles_tbl" lazy="true"
 */
public class UserRoleRelation {
    
    private UserRoleRelationID   identifier = null;
    private UserModel user = null;
    private RoleModel role = null;

    /**
     * @hibernate.id unsaved-value="any"
     */
    public UserRoleRelationID getIdentifier() {
        return identifier;
    }

    public void setIdentifier(UserRoleRelationID identifier) {
        this.identifier = identifier;
    }

    /**
     * @hibernate.many-to-one column="user_id"
     *            class="UserModel"
     *            not-null="true" cascade="all"
     *            insert="false"
     *            update="false"
     */
    public UserModel getUser() {
        ...
    }

    /**
      * @hibernate.many-to-one column="role_id"
      *            class="RoleModel"
      *            not-null="true" cascade="all"
      *            insert="false"
      *            update="false"
      */
     public RoleModel getRole() {
         ...
     }

     ...

}

You will notice the user of a class called UserRoleRelationID as our Hibernate identifier. This is due to the fact that XDoclet does not currently support composite primary keys. However, by defining our UserRoleRelationID class in the following way, we are able to get around this limitation and use our composite primary key:


public class UserRoleRelationID implements Serializable {

    private Serializable userID = null;
    private Serializable roleID = null;

    /**
     * @hibernate.property column="user_id"
     *    type="java.lang.Integer" not-null="true"
     */
    public Serializable getUserID() {
        return userID;
    }


    /**
      * @hibernate.property column="role_id"
      *    type="java.lang.Integer" not-null="true"
      */
     public Serializable getRoleID() {
         return roleID;
     }

    public int hashCode() {
        ...
    }

    public boolean equals(Object o) {
        ...
    }

    ...

}

Now that our Hibernate objects are defined, we need to create a Hibernate Query that accomplishes the same thing as our SELECT statement does. However, by working with Hibernate and utilizing our UserRoleRelation class, you will find the complexity of our query is greatly reduced. Our Hibernate Query can be defined as follows:


SELECT DISTINCT j.user AS p FROM UserRoleRelation j
   INNER JOIN j.user AS urj
   WHERE j.role.rolename IN (:roleList)
   GROUP BY j.user HAVING COUNT (j.role) = :roleCount

To execute this query, our Java command would look something like this:


/*
  * The query we defined above
  */
 String queryString = ...

/*
 * Declare a List containning the names of the roles we
 * wish to match on
 */
ArrayList roleList = new ArrayList();
roleList.add("role_a");
roleList.add("role_b");

Session s = ....
Query q = s.createQuery(queryString);
q.setParameterList("roleList", roleList);
q.setInteger("roleCount", roleList.size());

List users = q.list();

You will see that the results returned by our Hibernate Query are exactly the same as those returned by our SQL SELECT statement, but with a greatly reduced amount of complexity.

This type of scenario is very common when dealing with many-to-many relations. For example, a site that maintains user profiles and keywords describing those profiles. We could modify our code slightly to allow a user to search for profiles that contain the keywords "jazz" and "hamburgers". Another example would be an online shopping site. It's very easy to search for the keyword "dogs", or search for the keyword "books". Our approach now allows us to search for books about dogs. Additionally, we can modify our approach to allow for less refined matches. For example, modifying the last line in our query to read:

GROUP BY j.user HAVING COUNT (j.role) >= :roleCount

will allow us to match on a variable amount of roles. If we have a list of 25 roles, we could search for those users that are members of 10 or more by changing the roleCount variable to 10 instead of roleList.size().

As you can see, using Hibernate in our Java application makes performing these complex queries much easier than coding in raw JDBC.

JUnit Tests for Spring and Hibernate

2006-08-09T11:02:00.000-05:00

There are two big things that I love about the Spring Framework:

It has excellent Hibernate support through the use of its HibernateTemplate class
It makes writing JUnit tests much simpler, yielding better code

However, up until now I hadn't been able to find a good way to combine these two beneftis and JUnit test Hibernate-based code that I had written using Spring as my foundataion. (Mostly because I'd never really bothered putting any effort into figuring out the best way to do it) Up until now, I had my JUnit tests create a Spring ApplicationContext, and simply obtained my Hibernate DAO and ran my tests. While this did accomplish my goal of determining if my Hibernate-based code was working properly, it was poor design, and was terribly inefficient.

Recently, however, I came across a great article explaining how to unit test Hibernate mapping configurations. The article goes into how to create your session factory, and wire up your DAO with a new HibernateTemplate. It even gives tips on how to setup an in-memory HSQLDB database for testing. This reduces the overhead needed to perform your tests, as you don't need a full-fledged SQL database to run your tests against.

If you write any Hibernate-based code and are looking for a means of testing your Hibernate configuration, this is article is a great referrence.

OdyssiCS v0.1 Released

2006-07-28T11:03:00.000-05:00

At long last there is something coming out of my Odyssi CS project. Version 0.1 has just been posted on the project page. It is important to note, however, that this release should be considered alpha quality at best. A great deal of functionality has been left out of this release, and virtually all of the installation and configuration must be done manually. The main reason behind this release was just to get something out of the door. Some initial screenshots can be found here. The interface is pretty rough, and I've nearly completed a much more user-friendly interface that will be included in the next version. Version 0.2 will represent a pretty large redesign of the application, and should have much more functionality included.

At this point, the following is included in version 0.1:

Submit client certificate requests via a web browser (IE or Mozilla-based browsers)
Download an approved certificate into the client browser
Registration Authorities may approve or reject pending certificate requests

As you can see, the functionality is pretty minimal. In fact, the following are things that are not included in this release and are important to make note of:

Registration Authorities authenticate to the CA using a username/password combination managed by Tomcat (or whatever application server you have deployed to) -- NOT client certificates
Registration Authorities must have the "registrationAuthority" role in order to access the RA administration pages
No certificate extension support is included for any certificate generated by the CA, although this will be implemented very shortly. (There are a couple of bugs that need to be worked out)
No certificate revocation or CRL support is included
You may not currently specify which Java cryptography provider you wish to use
All installation and configuration must be done manually

The following libraries were used for this version of the project:

Struts 1.2.x for the web interface
Spring for dependancy injection and IoC
Hibernate for data persistence
XMLBeans for XML->Java mappings

Version 0.2 will maintain its use of Spring and Hibernate, but Struts may be replaced by another web application framework. In addition, I plan to use more of the design patterns found in Core J2EE Patterns and Core Security Patterns to create a more robust and secure application. I'll also be writing some articles about these patterns, outlining how they are being used in the application.

I have also released version 0.1 of the Odyssi ASN.1 library. This library provides the ASN.1 encoding for certificate and certificate extensions. It is extracted from the BouncyCastle JCE library, and has been modified to work with any Java crypto provider. A pre-build JAR is already included in the Odyssi CS distribution, so you will not need to build it separately. However, it is provided as a separate download for you, if you wish.

For those of you who are brave enough, Version 0.1 can be downloaded here. Make sure you read the INSTALL.txt and README.txt files for information on how to configure and install the web application. And, as always, please make use of the forums for any questions, comments, or criticisms you may have. Good luck!