JavaScript in the Data Center?

First I saw that PayPal was using Node.js for production services.  Then Airbnb.  What the hell is going on?

KrakenLogo

Well, a pattern is emerging:

  1. A new product or service team starts operating in lean-startup-style.
  2. This means rapid-prototyping so you can have quick build/measure/learn cycles.  There are a many choices for a rapid-prototyping stack, including: Ruby-on-Rails, Django, Node.js (and many more).  [Notice that Java/Spring/JSP – the current Data Center stack champion – is not one of them.]
  3. After piloting and iterating on the product/service, it’s blessed (usable, feasible and valuable) and it’s time to scale!  Now a decision is necessary: rebuild in the current de-facto enterprise stack (e.g. Java), or reuse/refactor the pilot code?  Well you can certainly reuse CSS style definitions, and if you used client-side templating then maybe you can re-use that, and then the inevitable question keeps poking it’s head: “Why not just use the pilot code/stack as the basis for the product?”

Now it’s very true that during build/measure/learn phase the prototyping engineer was optimizing for developer time.  That means the goal of the code was to learn about the problem and solution – not to follow best engineering practices.  As a consequence:

  • Not all use-cases / features have been implemented
  • Test coverage may be poor or non-existent
  • The code and database queries have not been optimized, and have been written for speed of development, not speed of execution
  • It has not been internationalized / localized / translated
  • It doesn’t necessarily support all required browsers / devices
  • There may be orphaned chunks of code
  • etc

But what if you bake-in as many best-practices into your prototyping stack?  Have a boilerplate rapid prototyping app with:

  • centralized logging (e.g. sumologic)
  • performance monitoring and analytics (e.g. statsd)
  • a deployment framework that can deploy the rapid-prototyping stack with load-balancing, auto-scaling, zero-downtime-deployment, etc. (e.g. aws, cloudformation, etc)
  • ready-to-go for internationalization
  • cross-browser-libraries loaded (e.g. jquery, etc).
  • build on a responsive-design framework (e.g. bootstrap3, etc)

There will still be orphaned chunks of code, un-optimized data access and manipulation patterns, strings to be translated, etc.  This is all manageable.  The biggest remaining hurdle to using the pilot-codebase as the foundation for the product is the technology choice, and the objections:

  • “That RoR/Django/Node.js stack is unreliable”
  • “It doesn’t scale”
  • “Who’s going to maintain it?”
  • “We can’t have 1,000,000 stacks in our services architecture”

Since there are examples of RoR/Django/Node.js stacks in production for large-scale products/services I think the first two objections are FUD.  But the second two are more legitimate.  And here’s what I’m seeing: companies are choosing and standardizing on a second-stack.  They’re opening up the service-architecture and data center and letting that stack play with the other kids.

I used to think that the momentum was in favor of Python / Django, but that was probably  due to personal experience and bias.  Now I think that Node.js might be the one.

FYI: Here’s that Bill Scott / PayPal presentation (from O’Reilly Fluent Conf 2013):

here are the slides (slideshare).

 

BONUS GIFT: Here is another great presentation on Node.js at PayPal.

 

About brendansterne

Director of Innovation Labs, Indeed.com
This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

One Response to JavaScript in the Data Center?

  1. for what it is worth, we see more momentum toward javascript than python. there was a good post about this that I read that was a corollary of the “power law” of programming languages – in the corollary, it is “anything that can be written in javascript will, eventually, be written in javascript” 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s