It’s been a busy few months. I’ve been busy at Totsy.com a startup that is taking the best of Lithium, MongoDB, and my time. If you want the lowdown of what I’m talking about check out the slides from Mitch Pirtle CTO at Totsy.

To the business at hand – Simple ACL with Lithium and MongoDB. Let’s get down to it!

The heart of this ACL functionality rests with the Li3 Router. With the Router you can connect a URL to a specific Class::action. This is handy when you want to have /login route to app\controllers\UsersController::login(). Of course, your routes can become more complicated and robust with a little regex goodness. Take a look at a default route for example:

    Router::connect('/{:controller}/{:action}/{:args}');

When going to the url http://yourapp.com/users/login/23423  then Li3 will map to the users controller, login action (or method) and pass the 23423 as the argument or input to that method. That’s all we really need to hookup a route and in turn give us the ability to setup ACL.

If there are no routes hooked up in your app/config/routes.php then requests will go nowhere. Without some default routes hooked up you’ll app will be sitting dead without anywhere to go. This is the perfect time to setup an ACL. How do we get this all going?

The next step is to call upon MongoDB. With MongoDB we can embed an array with the route and parameter into a mongo document object. I would embed this array with the user document that has the information used for login. This way, you can pull the entire document with one query from Mongo. Too simple to be true? Lets take a step by step approach:

1) Hook up your code in routes.php to loop through the document pulled from your user document:

    if (isset($session['acls'])) {
        foreach ($session['acls'] as $acl) {
             Router::connect($acl['route'], $acl['connection']);
        }
    }

Note that in my app I put the necessary information in the users session after they successfully login.

2) Embed the acls array as part of your user document schema. For example:

{
    "_id" : ObjectId("4c8d53e0ce64e5f449992400"),
    "created_date" : "Sun Sep 12 2010 18:27:44 GMT-0400 (EDT)",
    "email" : "test@lcs.com",
    "firstname" : "Test",
    "lastname" : "User",
    "password" : "b119670d125b0bd4271b25ce39fa166bc3bf79a0",
    "acls" : [
        {
            "route" : "/orders",
            "connection":"Orders::index"
        },
        {
           "route" : "/posts/view/{:item:[a-z0-9\-]+}",
           "connection":"Events::view"
        }
    ]
}

3) Do a happy dance because you’re done!!!

With a little admin work you can put together groups and more robust configurations for ACL.  At the end of the day all you really need to do is loop through an array of routes and connections and you’ll have the exact ACL you need to keep everyone in order.

Posted in PHP.

Mission: Find all the items in a users shopping cart that expired 3 minutes ago.

If you want to cheat just scroll to the last code block. ;) Now for the setup.

This is a common query for any ecommerce website that wants to keep things “fresh” or run some analytics. Private sales sites are fond of this because an item in a shopping cart means that a customer is holding up inventory. That of course is a bad thing. This is just one of MANY reason why searching by date range is important.

So how do we handle this with Lithium our RAD PHP 5.3+ framework and MongoDB? To answer this lets use the mission above as our basis and spit out some code. Since you didn’t skip ahead lets build up our example just a little.

Let’s start in a BaseModel that will hold our core code. We will first assume that we want to query for some standard time values. To get things started we can setup a human readable protected array:

	protected $_dates = array(
		'now' => 0,
		'-1min' => -60,
		'-3min' => -180,
		'-5min' => -300,
		'15min' => 900
	);
That times above are second that we will either add or subtract from the php method time(). Our calculation will basically be time() + $_dates['key']. The result of that calculation needs to be converted into a MongoDate object. This is important for several reasons. One good reason is that its easier for us to query using dates if Mongo knows we are looking for date. We may also save ourselves some trouble if more native MongoDB temporal calculations are added to the core.

Using the MongoDate object you can do things like greater than ($gt), less than ($lt) etc, etc and well etc. So how do we create this object? We need to instantiate the date with the Mongo Driver PHP MongoDate class. Since we are using static methods in our Model we’ll use the following code to create and return our MongoDate based on the name-key we pass the method. We can place this method in the BaseModel as well.
	public static function dates($name) {
	     return new MongoDate(time() + static::_object()->_dates[$name]);
	}
That done, we can now save and query for data. The code below demonstrates how we can add some created and expire MongoDates to a cart document. The really important part to glean here is that we are calling our static method to create the MongoDate. This method will live in our CartModel which can extend the BaseModel.
	public static function addDates($product, array $options = array()) {
		$product->expires = static::dates('15min');
		$product->created = static::dates('now');
		return static::_object()->save($product);
	}

Note: The MongoDate object will look something like “Thu Aug 05 2010 01:32:41 GMT-0400 (EDT)”. This throws a few folks off thinking its just a string.

Now for our mission. Don’t worry I’ll repeat it here to save you from looking: Find all the items in a users shopping cart that expired 3 minutes ago. How would we do that in the CartsController? All we need to do is call a static method in the CartModel that does the search and let it know what range we are looking for:

    $cart = Cart::search(array('time' => '-3min'));

We’re going to let our model do the heavy lifting to get the data.

	public static function search($params = null) {
		$time = (!empty($params['time'])) ? $params['time'] : 'now';
		$user = Session::read('userLogin');
		return static::all(array(
			'conditions' => array(
				'session' => Session::key(),
				'expires' => array('$gte' => static::dates($time)),
				'user' => $user['_id'])),
			'order' => array('expires' => 'ASC')
		));
	}

There is a bit going on here so lets break it down. First we are checking to make sure that the time is there and set it if it’s not. Then for the sake of finding a specific users cart item we doing some ID grabbing from the PHP Session. Next is the query part: Give me all the user cart items that are greater than X date.

Notice in the code above that we are calling static::dates($time) to fetch our MongoDate. This will let MongoDB in turn know that we are searching for a date and query for it appropriately. This helps to cut out all the epoch timestamp manipulation in our code to properly search for values.

Whew! Now you are an expert and have enough ammunition to use MongoDates to the full.

I’ve been meeting folks who are having a hard time understanding the practicality of a document database and dynamic schema. I often hear, “I’m just trying to wrap my head around the concept.”

Some of the main concerns are:

  • Shouldn’t we be normalizing?
  • What about joins?
  • What about ORM?

To help with the introduction and transition we’ll need to do a little unraveling than wrapping. Let’s kick off this short discussion with a literal example of why a document store (in this case MongoDB) works well.

Imagine for a moment that it’s time for your yearly visit to the doctor. You’ve been put on a few supplements to improve your health and the doctor is monitoring your progress. Time for a checkup.

You walk into the clinic and give your name. The secretary looks you up in their booking system and confirms your internal patient number. “There you are Mr. Solutions (Patient #000-00-0001). You’re right on time, please have a seat.” says the secretary. Since they have more than one Mr. Solutions its a good thing they have some sort of ID. The secretary goes to get your file as you wait patiently to be called in.

Let me clue you into something that happens behind the scenes. Having worked in a hospital I know the secretary goes into the back-office (database) and searches for the file #000-00-0001 (query). They only get one file. They don’t get a set of files that have a primary key and join them together. There are no joins. They don’t put your name/id number into a mapper and look you up.

If you took a peek into the back-office there are small files and big files. Some files have yellow sticky notes and others have blue. They are (indexed) by patient number. While a handful of files look very similar in structure (schema) others seem to be quite different. One way or another there is just one file per patient.

What’s the point? Welcome to a document database that works.

Many medical professionals made the choice  that a single file/folder/grouping of papers was sufficient to store your medical history. Some clinics are messy, others are neat, yet this simple system works for them. Back at the clinic it seems there is some information needed from a cardiologist that is in their files.

The secretary quickly calls and asks for a copy of the information. When he gets the copy he places it in your file.

You probably got the point a few paragraphs ago.

The same concept holds true with a document database. More often than not (yes there are exceptions) you’ll find that storing a majority of the pertinent data in a single document works well. What do I mean by a single document?

Taking our medical example, this is what it could look like if your medical history was digitized using the document database – MongoDB.

{
"_id" : ObjectId("4c537d4b82fd211170000000"),
	"name" : "Mr. LightCube Solutions"
	"date_created" : "Sat Jan 2 2010 21:32:58 GMT-0400 (EDT)",
        "DOB" : "Sat Jan 1 1970 00:01:01 GMT-0400 (EDT)",
	"billing" : {
		"description" : "Home",
		"telephone" : "",
		"address" : "43 Happy Lane",
		"address_2" : "",
		"city" : "Light Land",
		"state" : "NY",
		"zip" : "111222",
		"country" : "USA",
	},
	"medications" : [
		{
			"_id" : "4c537b3382fd21df6f040000",
			"name" : "Activase"
			"description" : " tissue plasminogen activator",
			"dose" : "100mg Vial",
			"date_started" : "Fri Jul 30 2010 21:32:58 GMT-0400 (EDT)"
		}
	]
	"appointments" : [
		{
			"date" : "Fri Jul 30 2010 21:32:58 GMT-0400 (EDT)",
			"summary" : "Started patient on Activase due to heart disease."
			"scheduled_checkup" : "Mon Aug 30 2010 11:00:00 GMT-0400 (EDT)",
		}

	]
	"conditions" : {
		High Blood Pressure,
		Diabetes,
		Excessive Vitamin D
	}
}

What would that document look like in a SQL database? I would guess 4 or 5 different tables requiring joins would be in place.

Why normalize all that valuable information?

If its not practical to do this in reality then why do it digitally?

Store your data in a way that is practical and dynamic. For instance, maybe Mr. Solutions will be put on some special program that will need to be monitored different than previous trials. Why build a new database just for that? Just adapt the schema and embed.

A document database offers the flexibility, speed and simplicity that we already live by in other systems.

Hopefully this brief trip to the doctor helped to clear things up. Now go take your MongoDB medicine and call me in the morning. :)

If you were like me you started throwing all kinds of files into MongoDB with GridFS. When you took a look at the db.fs.files collection you saw something like this for a document:

	
{
	"_id" : ObjectId("4c40affcce64e5e275c60100"),
	"filename" : "My First File.jpg",
	"uploadDate" : "Fri Jul 16 2010 15:16:12 GMT-0400 (EDT)",
	"length" : 55162,
	"chunkSize" : 262144,
	"md5" : "46aa378be7f6f1f3660efd7de5c1cbb6"
}

Did you see the MD5 hash? It’s there for a reason you know.

Since my PHP/MongoDB application has an administrative backend multiple people are loading up files. There is always a possibility that they will upload the same file. Of course this would be a very inefficient use of storage especially when the file is a video or picture.  That’s where the MD5 field in fs.files comes in handy.

In PHP you can use the md5_file() method to get the MD5 hash before you save the file to MongoDB. Running a findOne query using the md5 of your tmp file will let you know if  a document for that file already exists. If it does exist, then you’ll get back the fs.files document of the preloaded file. Then you can use the _id as a reference and don’t bother saving the file. Can you imagine all the money you save in storage fees on Amazon S3?

This is a very common and reliable way of doing things since byte for byte you know the files are the same. The sample script below is a snapshot of code in a Lithium application (Lithium is a new PHP 5.3+ framework). I’m basically running a findOne({“md5″ : “$md5″}) query:

	
protected function write() {
		$success = false;
		$grid = File::getGridFS();
		$this->fileName = $this->request->data['Filedata']['name'];
		$md5 = md5_file($this->request->data['Filedata']['tmp_name']);
		$file = File::first(array('conditions' => array('md5' => $md5)));
		if ($file) {
			$success = true;
			$this->id = (string) $file->_id;
		} else {
			$this->id = (string) $grid->storeUpload('Filedata', $this->fileName);
			if ($this->id) {
				$success = true;
			}
		}
		return $success;
}

Congrats 10gen on a great event. It was a huge cheer for MongoDB.

During and after the event you can see the strong acceptance of Mongo from the NYC tech community. There are several big names and budding start-ups that have already jumped right in with MongoDB and I don’t see this trend letting up anytime soon.

So why the blog? I gave the talk on MongoDB and PHP Development. Yes, it’s an exciting topic! Don’t you click away and give LightCube a high bounce rate.

Looking back, I took a real cursory path on both topics. That’s not a bad thing but I feel its time to dig a little deeper. If given the opportunity to speak again I’ll probably present more complex examples of how I and others have changed the “M” in LAMP to MongoDB. If you haven’t already clicked play jump to slide 23 and about 10:20 in the video for the “juicy” part. That’s where I talk about creating a simple blog with the RAD PHP Framework Lithium and MongoDB. Otherwise, don’t miss my Yankee Stadium reference and the Magento EAV whack in the first few slides.

If you wanted video of the whole event go here – http://mongodb.blip.tv/

Just a quick note to our readers that LightCube Solutions will be presenting at MongoNYC on May 21.

Go ahead, click the big badge above to register.

I’ll be speaking about PHP Development with MongoDB. There have been some really awesome projects launched in the PHP/MongoDB community and I’m going to do my best to bring those to the fore. If you don’t believe me check the list on github.

Our friends at 10gen were kind enough to extend a 25% discount to our readers. Please feel free to use the code ‘lightcube’ during registration. We look forward to see you there!

It was great attending and presenting at the MongoSF conference held on April 30th in San Francisco – Thanks 10Gen.

Speaking with a few attendees I got the feeling that many are moving forward with Mongo production deployments. I should be a bit more specific with my relative use of the word “many”:

“The Many” – Yes, I’m slightly jealous that the Ruby community was there in full force and vocal.

“The Other Many” – There were also folks who were jumping right in with Java and Python.

“The Not So Many” – I’m not getting the same feeling of enthusiasm from the PHP community.

The PHP community doesn’t seem as audible with their success/concerns/thoughts of using Mongo. There are developers using it and liking it. But it doesn’t “feel” as accepted yet. Maybe this feeling has some substance to it. According to a survey taken back in February 2010 the Ruby community dominated by representing 40% of the MongoDB users. We’re even outnumbered by Python users. Of course, one can question how far reaching this survey was. However, considering the fact that several PHP frameworks have made a concerted effort to work with MongoDB I’m wondering why I’m not hearing more about it. I’m probably not listening in the right circles.

With all that said and MongoSF in the past it’s time to prepare for MongoNY. I’m going to ask those aforementioned questions to the PHPers who show up so be ready! During the talk I’ll also discuss each framework and what it’s doing with MongoDB. Come to MongoNY and lets have a lively PHP/MongoDB discussion. You can use the discount code “lightcube” at checkout for a 25% discount.

In the meantime, here are my slides from the MongoSF presentation:

(Yes, my JSON business card on slide #2 validates!)

You’ve quickly put together the code to add files to MongoDB. But as any good DBA and Developer knows not everything that goes into your database should stay there.

Yes, you already figured out it’s as simple as $grid->remove(). Here is something to keep in mind. MongoGridFS::remove won’t present you with a message if it failed. To ensure that a file was successfully removed use MongoDB::lastError(). You’ll get an array in return that will show something along the lines of:

Array
(
[err] =>
[n] => 0
[ok] => 1
)

The field to watch is “ok”. The value of “1″ will return if the remove went according to plan. Otherwise you’ll get “0″ and it’s time to check the value for “err”. If you’re wondering about the “n” its the number of files that were removed.

In practice your code could look something like this:

/*
* function remove
* Removes files from Mongo and captures error if failed
* @return mixed
*/
public function remove($criteria)
{
    //Get the GridFS Object
    $grid = $db->getGridFS();

    //Setup some criteria to search for file
    $id = new MongoID($criteria['_id']);

    //Remove file
    $grid->remove(array('_id'=>$id), true);

    //Get lastError array
    $errorArray = $db->lastError();
    if ($errorArray['ok'] == 1 ) {
        $retval = true;
    }else{
        //Send back the error message
        $retval = $errorArray['err'];
    }

    return $retval;
}

With a few short lines you’re able to capture the error message and do whatever you like with it. Hopefully this helps keep your fs.files and fs.chunks collections clean.

As a quick followup to the last blog, I was asked to demonstrate how we could include some metadata with a file during or after upload.

This is a great feature since we can retrieve all the information attached with a document during a query. Including metadata (i.e. comments, dates, votes or WHATEVER ELSE YOU LIKE) is just as easy.

We are going to pick up the ball at the point where you’ve got all your MongoDB objects ready:

<?php
$date = new MongoDate();
//Use $set to add metadata to a file
$metaData = array('$set' => array(“comment”=>”MongoDB is awesome”, “date”=>$date));

//Just setting up search criteria
$criteria = array('_id' => $id);

//Update the document with the new info
$db->grid->update($criteria, $metaData);

?>

Did you break a sweat? Probably not.

BTW, I threw in a little extra tidbit there with MongoDate(). It’s a better way of inputting a date into Mongo instead of using PHP date with a bunch of formatting to get it just right.

I’ve been tweeting quite a bit about MongoDB over the past few weeks and its time to blog.

About 2 months ago we decided to install and mess around with MongoDB. We went from messing around to serious adoption about 2 weeks ago when we realized the power of working with it and PHP. It was Mitch Pirtle of www.spacemonkeylabs.com that first pointed us in this direction. Mitch was very enthusiastic about Mongo as he explained the great potential. Yet, it was only until I added an array of data from PHP into Mongo that my eyes started to open up.

Go ahead, open another tab in your browser (KEEPING THIS ONE OPEN) and Google MongoDB if you haven’t already (I made it easy providing the link). You’ll find sufficient documentation to give you the warm and fuzzy that 10gen isn’t messing around. They are in the database game for the long haul.

During our play with MongoDB we realized how true the term NoSQL is. For all you SQL grease monkeys out there it’s liberating. No Joins, No Select, just plain NoSQL.

As an example, let’s start off with putting uploaded files “directly” into MongoDB via GridFS (the storage specification for handling large files). Here is how simple it is:

First, we are going to assume you already have the code in place to handle the upload itself. We first tested this using SWFUpload which provides quite a bit of flexibility if you like to control how your page looks during upload. From a PHP perspective, the uploaded file will be accessible via the predefined variable $_FILES.

Here is what you have to do to get the upload into MongoDB:

<?php

$m = new Mongo(); //connect
$db = $m->selectDB("example"); //select Mongo Database

$grid = $db->getGridFS(); //use GridFS class for handling files

$name = $_FILES['Filedata']['name']; //Optional - capture the name of the uploaded file
$id = $grid->storeUpload('Filedata',$name) //load file into MongoDB

?>

Yep, that was it. One thing that took me a “second” to realize is that you actually pass the literal name of the file_post_name (in the example ‘Filedata’) to MongoDB. It does all the heavy lifting of getting the data from the system and storing it. Also, take note that you get an $id back which is the MongoID that acts as the “primary key”. That makes it easy for you to reference the file right away if you need to.

So that’s a rather quick look and tip for MongoDB. Stay tuned  as we continue to put out our MongoDB tricks and tips for PHP. We’re in it for the long haul too.