I’ve been meeting folks who are having a hard time understanding the practicality of a document database and dynamic schema. I often hear, “I’m just trying to wrap my head around the concept.”

Some of the main concerns are:

  • Shouldn’t we be normalizing?
  • What about joins?
  • What about ORM?

To help with the introduction and transition we’ll need to do a little unraveling than wrapping. Let’s kick off this short discussion with a literal example of why a document store (in this case MongoDB) works well.

Imagine for a moment that it’s time for your yearly visit to the doctor. You’ve been put on a few supplements to improve your health and the doctor is monitoring your progress. Time for a checkup.

You walk into the clinic and give your name. The secretary looks you up in their booking system and confirms your internal patient number. “There you are Mr. Solutions (Patient #000-00-0001). You’re right on time, please have a seat.” says the secretary. Since they have more than one Mr. Solutions its a good thing they have some sort of ID. The secretary goes to get your file as you wait patiently to be called in.

Let me clue you into something that happens behind the scenes. Having worked in a hospital I know the secretary goes into the back-office (database) and searches for the file #000-00-0001 (query). They only get one file. They don’t get a set of files that have a primary key and join them together. There are no joins. They don’t put your name/id number into a mapper and look you up.

If you took a peek into the back-office there are small files and big files. Some files have yellow sticky notes and others have blue. They are (indexed) by patient number. While a handful of files look very similar in structure (schema) others seem to be quite different. One way or another there is just one file per patient.

What’s the point? Welcome to a document database that works.

Many medical professionals made the choice  that a single file/folder/grouping of papers was sufficient to store your medical history. Some clinics are messy, others are neat, yet this simple system works for them. Back at the clinic it seems there is some information needed from a cardiologist that is in their files.

The secretary quickly calls and asks for a copy of the information. When he gets the copy he places it in your file.

You probably got the point a few paragraphs ago.

The same concept holds true with a document database. More often than not (yes there are exceptions) you’ll find that storing a majority of the pertinent data in a single document works well. What do I mean by a single document?

Taking our medical example, this is what it could look like if your medical history was digitized using the document database – MongoDB.

{
"_id" : ObjectId("4c537d4b82fd211170000000"),
	"name" : "Mr. LightCube Solutions"
	"date_created" : "Sat Jan 2 2010 21:32:58 GMT-0400 (EDT)",
        "DOB" : "Sat Jan 1 1970 00:01:01 GMT-0400 (EDT)",
	"billing" : {
		"description" : "Home",
		"telephone" : "",
		"address" : "43 Happy Lane",
		"address_2" : "",
		"city" : "Light Land",
		"state" : "NY",
		"zip" : "111222",
		"country" : "USA",
	},
	"medications" : [
		{
			"_id" : "4c537b3382fd21df6f040000",
			"name" : "Activase"
			"description" : " tissue plasminogen activator",
			"dose" : "100mg Vial",
			"date_started" : "Fri Jul 30 2010 21:32:58 GMT-0400 (EDT)"
		}
	]
	"appointments" : [
		{
			"date" : "Fri Jul 30 2010 21:32:58 GMT-0400 (EDT)",
			"summary" : "Started patient on Activase due to heart disease."
			"scheduled_checkup" : "Mon Aug 30 2010 11:00:00 GMT-0400 (EDT)",
		}

	]
	"conditions" : {
		High Blood Pressure,
		Diabetes,
		Excessive Vitamin D
	}
}

What would that document look like in a SQL database? I would guess 4 or 5 different tables requiring joins would be in place.

Why normalize all that valuable information?

If its not practical to do this in reality then why do it digitally?

Store your data in a way that is practical and dynamic. For instance, maybe Mr. Solutions will be put on some special program that will need to be monitored different than previous trials. Why build a new database just for that? Just adapt the schema and embed.

A document database offers the flexibility, speed and simplicity that we already live by in other systems.

Hopefully this brief trip to the doctor helped to clear things up. Now go take your MongoDB medicine and call me in the morning. :)

Congrats 10gen on a great event. It was a huge cheer for MongoDB.

During and after the event you can see the strong acceptance of Mongo from the NYC tech community. There are several big names and budding start-ups that have already jumped right in with MongoDB and I don’t see this trend letting up anytime soon.

So why the blog? I gave the talk on MongoDB and PHP Development. Yes, it’s an exciting topic! Don’t you click away and give LightCube a high bounce rate.

Looking back, I took a real cursory path on both topics. That’s not a bad thing but I feel its time to dig a little deeper. If given the opportunity to speak again I’ll probably present more complex examples of how I and others have changed the “M” in LAMP to MongoDB. If you haven’t already clicked play jump to slide 23 and about 10:20 in the video for the “juicy” part. That’s where I talk about creating a simple blog with the RAD PHP Framework Lithium and MongoDB. Otherwise, don’t miss my Yankee Stadium reference and the Magento EAV whack in the first few slides.

If you wanted video of the whole event go here – http://mongodb.blip.tv/

Just a quick note to our readers that LightCube Solutions will be presenting at MongoNYC on May 21.

Go ahead, click the big badge above to register.

I’ll be speaking about PHP Development with MongoDB. There have been some really awesome projects launched in the PHP/MongoDB community and I’m going to do my best to bring those to the fore. If you don’t believe me check the list on github.

Our friends at 10gen were kind enough to extend a 25% discount to our readers. Please feel free to use the code ‘lightcube’ during registration. We look forward to see you there!

Where did January go?

Where did 2009 go?

Our last blog post was in December?

Wow, time is flying by. The good news is we’ve been quite busy here at LightCube. The bad news is we are breaking the rule of blogging regularly (sorry).

Looking back, the past two years has brought its challenges and lessons. But we are now looking forward and gearing up for what’s in store. Speaking of which, where is what we’ve been up to:

Tienda! – Who says the internet doesn’t need another eCommerce tool? If you check with the Joomla community you’ll quickly see there is a massive need. We’ve been working with Dioscouri to bring the first native eCommerce application to Joomla! The really exciting news is that Tienda v0.2.0 (Alpha) is available.

MongoDB – For a SQL Grease Monkey like myself this is a huge redirection. Don’t worry, we’re not abandoning our traditional LAMP stack just yet. It’s not so much about leaving MySQL as much as it is about being liberated to use something else. We’re already working on a few pet projects to use MongoDB for our core ACL and automatic MediaRSS feed generating.

WebEDI – What is EDI you ask? It’s a LONG story. But I can tell you this much,  it has been around before the internet and isn’t going away anytime soon. We are helping to enhance the EDI back office systems of companies that are warming up to web services and XML.

I did say a quick summary didn’t I? Although tomorrow isn’t promised to anyone this year is going to be a busy one. We’ll keep you posted (hopefully in increments shorter than a month).

Knock, knock. Let me in!

Port-knocking has been around for a while, so many of you may already be familiar with the idea. But for those who aren’t, I’ll take a minute to briefly describe it. The concept is that a service (daemon in the Linux world) listens on the link layer of your network interface for “knocks”, or small packets hitting a combination of ports that you have predefined. The ports don’t have to be open since all of the magic here happens at a lower level. Once the combination of ports have been “knocked”, the service responds by doing what you configured it to do, run some command on your system.

One of the most common uses involves opening and closing port 22 in your firewall for remote shell access. As to why someone might want to do this, try opening up port 22 on a public machine and then watch your logs fill up over the next few days with script-kiddies attempting to brute force their way into your system. If you have one user with a common username and weak password, it’s only a matter of time before someone breaks in.

To avoid this, you can set up a port-knocking server, like the one at zeroflux.org. The page there gives you configuration examples for exactly the scenario I related above. It even offers a mechanism to somewhat ‘randomize’ the ports being knocked so that someone listening on the wire can’t knock the same ports with the same results.

How do you use port-knocking?

Because the knock daemon can be configured to run any command upon the right sequence of ports, the possibilities for its use are quite wide. My question to you is, apart from the above example (opening up port 22 for shell access) in what other scenarios would you find port-knocking useful? Do you currently use it for automating other tasks? Comments welcome.

I delayed installing Windows SP3 on my MacBook Pro BootCamp partition because of some sub-conscience fear. If you think it will happen, it very well could – Parallels couldn’t get the Windows Virtual Machine to boot.

When I launched parallels the following screens showed up:

Windows Error #1

This was quickly followed by this screen:

Parallels_Error #2

Then it starting looping in those two screens. Never before have I seen a looping blue screen of death! After a brief investigation (translation: “low level panic attack”) and a few “force kills” in Leopard I found the solution to the problem on the Parellels Forum. For the masses of you who may already know about the solution: Great! For the rest who have been searching and haven’t found the solution this worked for me:

  1. Select XP Boot Camp Virtual Machine but don’t start it.
  2. Click “Hard Disk 1″ and open Configuration Editor.
  3. Click “Advanced” on the right panel.
  4. Click the “Clear” button to remove Parallels information from the boot camp partition.
  5. Start the Virtual Machine.
  6. When asked to Activate Windows say no.
  7. Activate windows AFTER it has fully loaded.
  8. Reinstall Parallel Tools.

It’s often the case in development you’ll find yourself embedding brand new code with the old. The simplest of commands can go a long way to making the content exchange easier. The code window.opener.document.getElementById, is a great way to pass information between webpages (the old and the new) that need to coexist. The scenario could be passing data between a parent window and child or using hidden popups. Here is a PHP example to update checkboxes on a parent window (or launching page) echoing javascript code.


//Initialize an array of checkboxes
$array_checkbox = array("chk_name1", "chk_name2", "chk_name3", "chk_name4");
$i = 0;
while($i<=4){
    $chkname = $array_checkbox[$i];
    echo "<script type=\"text/javascript\">";
    echo "window.opener.document.getElementById('$chkname').checked=true; </script>";
    $i++;
}

Wasn’t that quick and easy! You’ll notice that you could just as easily replace the checkbox names with div’s, radio buttons or anything that needed to have its value/property modified. This opens a world of possibilities for you doesn’t it?
 

This is really just an informational post, and a test of sorts. We’re moving our blog from Blogger.com to a locally installed version of WordPress. There have been a number of articles written about Blogger vs. WordPress, and I’m sure if you Google it you can find all sorts of views on the subject. When it comes down to it, it’s really just a matter of perspective. We chose to move to WordPress for the following main reasons:

  1.  More editing functionality and possibilities.
  2.  Easier template editing (in my opinion).
  3.  Larger variety of plugins
  4.  Completely installed and managed locally instead of publishing content to a local address through FTP.

If I wanted this to be a long post, I’m sure I could continue to find and list reasons. Suffice it to say that we’re now using WordPress and the URL for the site has changed. From now on, you can access us here: http://www.lightcubesolutions.com/blog/

In response to the follow-up on the “Flock” browser, I decided this post was necessary. You see, I too felt that the browser was too busy; almost like I needed ADHD just to process the shear amount of information being delivered! (As an aside, I think I’m going to trademark the term “informatsunami™”: the farther back a browser/feed goes, the more overwhelming amounts of data get returned…with no escape!) The painful reality of my discomfort with using the browser in its intended state hit me with force of, well, a tsunami: I’m older now, I don’t need all that “stuff”!

 

Perhaps it really is a sign of maturity, know quite matter-of-factly what I want and how I want it delivered to me. I could care less if my cell phone has a 2.0 megapixel camera with zoom, or that it plays mp3 files, or that I can download a new ringtone. I JUST WANT TO MAKE A PHONE CALL FROM SOMEWHERE OUTSIDE MY HOUSE AND NOT WORRY ABOUT FINDING A PHONE TO USE!!!! (whew, for a minute there, I lost myself…I lost myself…) Where was I…ah yes, how I want content delivered to me, I digress. I think it’s “cool” to have everything in one tidy Mozilla-based browser. But I really don’t want everything. I don’t want to be part of a social network online; it’s hard enough maintaining my relationships in person! I don’t want to share my pictures with the world (or with the select few people whom I grant access), nor do I want to view everyone else’s pictures. I don’t want to write a blog everyday, as indicated by this being my first blog post in about 5 months! I do want to check my email, the few RSS feeds I subscribe to, and some blogs/articles that center on my current professional activities; I may even want to play a game or two online. Of course, I realize that I could simply tweak/customize Flock’s settings to behave in manner more fitting my discriminating tastes. But then I realized something else: I don’t want to. I joked with “JH” about being too old for Web 2.0, about being passed by. That used to make me sad, but really, I feel liberated.
Getting to the title of this post, my memories too me back in time about 10 years ago. I was a young(er), cocky programmer who had just cut his teeth on a Y2K project, learning Fortran, TAL, C, COBOL and statistical analysis in 18 months. I was moving on to a small consulting firm where I’d learn VB/SQL/ASP development, along with Oracle, Sybase, Linux development/administration. I was barely old enough to drink, yet I had surpassed the technical experience of every person in my family. I had a cell phone! I knew every punctuation combination used to create a smiley! I knew every IM acronym! I downloaded mp3 files before it was illegal! (Ok, ok, it was always illegal; but that was when few got in trouble for it…) I read “journals” updated daily online. The fact is, I’ve already been there, done that. I don’t care anymore, or least I don’t care as much what’s new and improved: it’s really old and declining…or maybe I am. Either way, I might still use Flock and whatever else new comes along. I might even get swept up in the informatsunami™ (remember you read it here first!) But in the end, I’ll still pine for the “old days (you guessed it)…when we were kings…

This is a test. If this post is successfully published, it means I am currently using one of the most useful and complete web browsers of the Web 2.0 age.

Meet Flock. Flock brings together your online presence into one complete package. Using sidebars and widgets, flock connects you to your Gmail, Yahoo Mail, AOL Mail, Flickr, YouTube, Digg, Facebook, Picasa, Blogger.com, (etc., etc.) accounts and lets you access/use their features within one sleek interface. The actual core browser is powered by Mozilla, which means that if you know or use Firefox, Flock will feel very familiar.

Everyone’s been talking about Web 2.0, bringing the internet to life and exploring new possibilities with dynamic content and interaction. Well, here it is.

Blogged with the Flock Browser