Jason Leveille's Blog

Last week Dave and I were trying to figure out the best way to synchronize member data from an asp application with a WordPress installation. I don’t remember how it came up in conversation, but in a flurry of chatter we decided that we should build a RESTful API to allow the applications in question to communicate. In the interest of re-usability we also decided that we should build a generic toolkit. One that could easily be used to expose an API for any supported database, through PHP. So, using this toolkit you should quite easily get your Lasso application talking with your PunBB software, if that’s what you need.

So was born rest-toolkit. We have released an alpha version of the project, so download it and give it a try. I hope you don’t mind a lack of formal documentation. Depending on how things go documentation will follow.

The quick install guide

• Find a PHP project you have in the works that talks with a database
• Make sure you don’t care about the data in the database, especially if you’re going to be using DELETE
• Unzip the rest-toolkit into a directory (perhaps /api)
• Open config/config.php and make any changes that need to be made
• Modify the database connection information in database/mysqli.php. If you want to use regular old mysql you’ll have to make some additions to the mysql.php file (take a look at the iDatabase interface file for some clues)
• Do you have a users table in your database? If so, visit http://your.url/restapidir/users/, or http://your.url/restapidir/users.xml, or http://your.url/restapidir/users.json, or http://your.url/restapidir/users/1.xml, or http://your.url/restapidir/users/1.json.
• Take a look at the class.Blocks.php example file for an example of an extended resource file.
• Of course, before using this in anything even remotely resembling production you would need to protect your API directory and require authentication to access the REST resources.  That part is left up to you.

Some Highlights

• When you make your REST request the query will be built dynamically based on the field names you send with the request. So, 95% of the work is already done for you.
• Include an xml or json extension with your request to modify the response type. Need something other than xml or json? You can easily add other extensions and mime-types.
• Want a clean url, but your clean resource name doesn’t nicely match a database table? Don’t worry, you can extend the resource base class with a class for your clean resource name (for example, class.Users.php), and map the resource to any table in your database. By default the resource name you include in your url will map to a table of the same name.
• Need to limit what is returned/updated/created in a query? These items are easily configurable in your extended class.
• Need to limit the resources that your API can talk with? In the resource base class you can define the resources the API should expect to receive requests for.

Tagged in : REST, Web Services

Recently I worked on a project where all member data was being managed through a third party member management service, provided by a company named Avectra. The product does more than manage member data from the looks of things, but all I cared about was interaction with the member management API.

I had to authenticate against an external data source, as well as query the data source for member information when needed. Avectra does have a web service API which it has named xWeb. Here is a quote from their Wiki:

xWeb is netFORUM’s XML web services application. netFORUM is an enterprise level Association Management System (AMS) developed by Avectra that allows associations to manage their customers and related activities. netFORUM is used by association staff, members, and the public at large.

Interacting with the API

After a successful authentication with Avectra, the authenticated user is presented with a token that they pass along with API requests. Pretty standard stuff. The Wiki does an ok job of explaining authentication and login (two different things), therefore I’ll jump right to the good stuff. If you’re stuck on authentication/weblogin please let me know.

For interacting with the API I created a class file to abstract, at least to a small degree, some of the details of consuming the service. Here it is:

<?php

/*
 * @author: Jason Leveille
 * Provides access to avectra web services
 */

class Avectra
{

   //Avectra organization api username and password (not member username/password) to access web services
   private $user = 'user';
   private $pass = 'password';

   //Avectra paths to xweb and eweb.  This is handy so that you can easily switch back and fourth between dev and production
   private $xweb = 'www.mencnet.org/xweb/';
   private $eweb = 'https://www.mencnet.org/netforummenctest/eweb/';

   /**
   * Grabs data associated with a member based on customer key / token
   * @param $cst_key Customer Key to search on
   */
   public function getMemberDataByKey($cst_key) {

      if(empty($cst_key))
      {
         return false;
      }

      $memberData = sprintf('https://%s:%s@%snetFORUMXML.asmx/GetIndividualInformation?IndividualKey=%s', $this->user, $this->pass, $this->xweb, $cst_key);

      //The simplest solution is to call simplexml_load_file directly, however this was causing issues on Solaris 10, php 5.2.  Didn't have time to investigate, so I went with load_string
      //return simplexml_load_file($memberData);

      $data = simplexml_load_string(file_get_contents($memberData));

      //make sure that the returned xml file actually contains data
      if(empty($data->IndividualObject[0]->eml_address))
      {
         return false;
      }

      return $data;

   }

   /**
   * Grabs data associated with a member based on member email
   * @param $email Member email to search on
   */
   public function getMemberDataByEmail($email) {
      if(empty($email))
      {
         return false;
      }

      $memberData = sprintf('https://%s:%s@%snetFORUMXML.asmx/GetQuery?szObjectName=%s&szColumnList=%s&szWhereClause=%s&szOrderBy=%s',
         $this->user,
         $this->pass,
         $this->xweb,
         'Individual',
         'ind_first_name, ind_last_name, cst_org_name_dn, adr_city, adr_state, eml_address',
         sprintf("eml_address = '%s'", $email),
         "ind_last_name"
      );

      return simplexml_load_file($memberData);
   }

   /**
   * Grabs the url of the avectra eweb application
   */
   public function getEwebUrl() {
      return $this->eweb;
   }
}
?>

So, for example, to retrieve data about a specific member, and to store that data in session variables for later use, your code might look similar to the following:

function login_success($cst_key = null) {

   if(!$cst_key)
   {
      //set error message and redirect
   }

   //ensure clean referer data
   $original_referer = $this->clean->stripScripts($this->params['url']['ref']);

   //Avectra object
   //include avectra class file, for example, in CakePHP
   App::import('Vendor', 'Avectra');
   $avectra = new Avectra();

   //ensure a clean customer key
   $cst_key = $this->clean->stripScripts($cst_key);

   //if we successfully load the associated user data
   if($data = $avectra->getMemberDataByKey($cst_key))
   {
      $data = $data->IndividualObject[0];

      $this->Session->write("Member.memberId", $this->clean->stripScripts($data->eml_address));
      $this->Session->write("Member.firstName", $this->clean->stripScripts($data->ind_first_name));
      $this->Session->write("Member.lastName", $this->clean->stripScripts($data->ind_last_name));
      $this->Session->write("Member.email", $this->clean->stripScripts($data->eml_address));
      $this->Session->write("Member.location", $this->clean->stripScripts($data->adr_city . ', ' . $data->adr_state));
      $this->Session->write("Member.loggedIn", 1);

      $this->Session->write("Member.customerKey", $cst_key); 

      //indicate successful login and redirect to original referrer

   }
   else
   {
      //set error message and redirect
   }
}

The SOAP Alternative

The Avectra Wiki examples are slanted heavily in the direction of SOAP, however in this instance I just thought it was overkill. I figured the response was going to come back in XML, so I would just use simplexml_load_file and pass in the URI. The application lives with Solaris 10 and PHP 5.2. For some reason simplexml_load_file was failing under certain boundary cases that I couldn’t quite pin down. This was an easy fix as I just used simplexml_load_string and file_get_contents to retrieve the xml response. An extra step, but it seemed to solve whatever the issue was that I was having. So, all I needed to do was make sure that I had the properly constructed URI, and that I could authenticate. Authentication is simply handled by sending in the username:password combination as part of the request.

Tagged in : PHP, Web Services

It seems that more and more people are moving away from the inclusion of www in their url. SitePoint recently wrote about the subject, and this past February Roger Johansson also wrote an article on the topic. Roger’s article was really more about making sure your site works with our without www (I couldn’t agree more). You wouldn’t get any more from me than you could get from taking a look at the articles I’ve referenced, therefore I’ll just skip to the good stuff. In general I feel that the inclusion of www is just a waste of address bar space. For this blog I have decided to throw it to the dogs. If I remember correctly, “Removing the Ws from URLs“, an article from 2002, was a big help for me on my path to www enlightenment.

Removing WWW through VirtualHost

<VirtualHost ipaddress>
ServerAdmin root@localhost
ServerAdmin webmaster@blueatlas.com
DocumentRoot /path/to/public_html/
ServerName jasonleveille.com
ErrorLog /path/to/logs/jasonleveille.com/error_log
TransferLog /path/to/logs/jasonleveille.com/access_log
</VirtualHost>

<VirtualHost ipaddress>
ServerName jasonleveille.com
ServerAlias www.jasonleveille.com
ServerAlias ww.jasonleveille.com
Redirect permanent / http://jasonleveille.com/
</VirtualHost>

Removing WWW through .htaccess

Perhaps you don’t have access to your VirtualHost block. Not a problem if your host/server/whatever allows you to override directives via .htaccess. This article on using .htaccess from Daring Fireball is a good reference regarding this topic.

RewriteEngine on
RewriteBase /
RewriteCond %{HTTP_HOST} ^www\.yourdomain\.com$
RewriteRule (.*) http://yourdomain.com/$1 [R=Permanent]

Tagged in : Apache, Server Administration

I recently had the need to set up some permanent redirects for an application built on CakePHP. My first thought was to use a simple .htaccess Redirect at the top of my www .htaccess file, like such:

Redirect 301 /foo /bar

However, the result of the redirect was actually the following url:

http://foobar.com/bar?url=foo

This has to do with the QSA flag in the last RewriteRule of the root htaccess file. The QSA flag indicates that a query string should be appended to the url. In our case, foo is recognized as the query string and it ultimately is appended at the end of the bar redirect. Not what I wanted, as the appending of foo to bar results in a 404 error.

Using a RewriteRule

My next thought was to set up a RewriteRule for foo, so that it would be redirected to bar. Here is what I came up with, with its exact placement in the default CakePHP webroot htaccess file. The really important thing here has to do with the placement of the rule:

RewriteEngine On

#rule to redirect foo to bar
RewriteRule ^foo$ /bar [R=301,L]

RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ index.php?url=$1 [QSA,L]

As you can see here, I am using the R and L flags at the end of the RewriteRule. These flags serve the following purpose:

R: Forces a redirect to the new url, and loads the new page
L: Ensures that this is the last rule that is processed by the server in this file. This is important here, as without the L flag the server would move on to the last rewrite rule, essentially ignoring the R flag for our redirect rule, and it would end up generating a 404 error.

Here’s a great set of cheatsheets if you’re interested in reading more.

Other options

Of course there are other options for redirects, including meta refresh or PHP header, however I didn’t want to clutter up the directory with pages whose only purpose was that of setting up a redirect.

Tagged in : CakePHP, Mod Rewrite

The death of an open source project starts when the lead developers stop actually using the project. If it is only being maintained out of a feeling of obligation for those who have downloaded and installed the code, than it’s bound to have a short shelf life. In July of 2006 Stephen Eskin (a former student of mine) and I launched the first version of Project Alumni. We had implemented an alumni application for our High School and we figured we might as well make it available to the world. In nearly two years the project has been downloaded approximately 3400 times. I’m pretty happy with those numbers. Just query google for “Powered by Project-Alumni” and you can get a good idea of the number of sites using the project.

Personally

On a personal level I’ve always been happy with the responses we’ve gotten from the people using the project. For what it’s worth, I’m proud of the fact that this was the first real application I ever took part in developing. I’ve always been really impressed by what Eskin was able to accomplish with almost no PHP experience, and very little development experience. The project really has it’s origins in the need for a teacher to find something for a very bright student to do. We launched the first Project Alumni 1 month after conceiving the project in October of 2005, and most of that credit goes to Eskin.

Professionally

On a professional level I’ve never felt quite right about the codebase. Considering the inexperience of Eskin and I when the project was first open sourced, and considering how much I didn’t know about basic security best practices, I think it’s ok. Ok won’t cut it though when new security vulnerabilities are discovered and I’m not around to patch them. Even though the data we’re housing might not be that sensitive (others might argue otherwise), a member might be using the same username/password that they use for their banking account. If that data isnt’ safe we have a problem. At least we are using mysql_real_escape_string and hashing the password. Actually, the more I dig into the code (again, which I have been disconnected from for a while now) the more I see things that I like. For instance, checking for ‘magic_quotes_gpc‘ and stripping slashes before running mysql_real_escape_string. Good one whoever implemented that way back in 2006. Someone was paying attention to the PHP manual. Probably Eskin.

Support

I basically stopped supporting the project 4 months ago. The only reason I made a commit was because someone started asking me about security vulnerability reports that had started showing up. So, I pushed out a patch and moved on. I used to maintain a website and a help forum, but I decided one day that I didn’t want that burden anymore, especially with a codebase that I wasn’t really invested in. Don’t get me wrong. I used to be invested in the project. I spent 8 hours on my day off once getting the application to install properly and behave on a Windows server running IIS, just because the guy who needed help was a nice guy.

Not Using your own Code

So, how do you kill an open source project? I have stopped releases. I have stopped supporting the project. I have stopped submitting patches. I took down the website and forums. Still, this thing is getting downloaded 100 - 200 times a month. I feel like I can’t just pull the plug. I think the real way you kill an open source project is by not actually using it. I think the best projects are the ones where the lead developers are heavily invested in the security/structure/scalabilty/maintainability of their own code. It was easy to use this project when I was teaching and I was responsible for the websites at Quince Orchard. A security fix was good because it directly impacted a site that I was responsible for. That’s obviously changed now. An up-and-coming open-source software project of mine on the other hand will see a lot of use in the future. A small percentage of our client base at BlueAtlas is a perfect fit for what SimpleMC has to offer. Another percentage of our client base is a good fit after some client specific customization of the application. Some of which makes it back into the project, and some of which does not.

So, to kill or not to kill, that of course is the question. I’m definitely leaning towards no now that I have written this post. I was way in the direction of yes 30 seconds before I started writing. I would be interested to hear what others think.

Tagged in : Open Source

I’ve been working on a project for a while now at work which we have dubbed SimpleMC (Simple Managed Content). I spoke with Dave this morning, and he gave me the go ahead to start the process of open sourcing the project … which I’m very excited about. I haven’t officially even released an alpha version yet, however you can get a sneak peak:

• Google Code Repository
• Live Demo
  • u: admin
  • p: admin

I apologize in advance if someone gets in there and changes the admin password or completely destroys the demo.  It is a demo and thus I haven’t spent a lot of time (5 minutes) putting it in true demo mode.  I’d also like to mention that you won’t find any documentation about getting SimpleMC to work for you.  You can take a look at /bai_cms/demo/ in the source to see how it is integrated into a page.  Any issues, just contact me or post a comment here.

Tagged in : Open Source

I recently implemented DFS and Iterative Deepening algorithms to search for a target node in a tree. Initially I thought I would write my program in C++, however after some thought I decided to go the route of JavaScript. I believe the source code is commented sufficiently to give you a glimpse into some of the decisions I made, therefore I won’t be providing any real detail in this post.

DFS and Iterative Deepening Project

I have to give much thanks to Kirupa, as his ActionScript implementation of DFS and BFS gave me the inspiration to go the route of JavaScript. Not only that, but I was able to use much of his work as a model for the direction I took. I hope you find that I did provide sufficient credit in my comments. Enjoy.

NOTE: I stayed up until 1am, two nights in a row to finish this. I don’t work well late at night and that is likely reflected in some of the algorithmic decisions I made. If you have any questions please don’t hesitate to ask. Also, you might be asking, WTH did he use extjs here? Honestly, I don’t know. I originally had grand visions for my node tree, etc, however those didn’t pan out. In it’s current state there is no advantage to using that framework.

Tagged in : JavaScript, PHP