Friday, October 18, 2013

Reusable javascript modules with Bower

In a previous post I have used require.js to download javascript files and optimize them.
Here I'll introduce a useful tool to manage your javascript and css assets. But first of all let's talk about modules definition in Javascript.

Universal module definition

At the moment there is no standard way to define a module (we are waiting for EcmaScript6) but you can easily define your module in a way compatible with AMD, UMD and regular script import. Just pick one of these snippets. I recommend to use one of those because declaring object in the global namespace is not a good practice and libraries like requirejs can help to load and optimize your scripts.

Bower

Bower is a package manager designed for the web. You can install it using:

npm install -g bower

And then start downloading your first package (jquery for example):

bower install jquery

You'll find jquery in the jquery folder inside the "bower_components" folder. Bower packages are versioned so you could (and should) specify what version install.
bower install jquery#1.10.2
If this package has some dependencies (other bower packages of course) those will be installed in the same "bower_components" folder. For those who are using node.js: be aware that it is quite different from how npm works! In this case all the dependencies are resolved and flatten in the same folder.

Bower package

A bower package, in fact, is a simple folder with a bower.json file:
{
  "name": "mypackage",
  "version": "0.0.1",
  "main": [
     "script.js",
     "style.css"
  ],
  "homepage": "https://github.com/User/myproject",
  "authors": [
    "Donald Duck "<donald.duck@gmail.com>";
  ],
  "description": "an example package",
  "keywords": [
    "js",
    "css",
  ],
  "license": "private",
  "private": true,
  "ignore": [
    "**/.*",
    "node_modules",
    "bower_components",
    "test",
    "tests"
  ],
  "dependencies": {
    "lodash": "~1.3.1",
  }
}
Let's see the most important fields:

name: the name of the package

version: the version of the package. It should be the same of the tag on git.

main: these are the main files of your project. For example a jquery repository has a lot of stuff (unbuilded file, test, fixtures etc.) but the main file is just jquery.js.

dependencies: an array of bower packages needed to run this one. You should always specific the version using this syntax.

Bower repository

The published packages are just git tags with a bower.json. The bower command  anyway uses a central repository to store the metadata (name, url of repository, keywords etc.). This repository is on https://bower.herokuapp.com/. If you have a public project you can publish on the central repository using:
bower register mypackage https://github.com/user/mypackage.git
The versions available will be taken from the available tags on you repo.

Running your own repository

Let's say you are working on a project and you don't want to share every package you are working on.
You can easily set up your own bower registry and configure bower to use it. You can do this putting a ".bowerrc" in your working directory (or in your home):
{
    "registry": {
        "search": ["http://bower.myserver.com", "https://bower.herokuapp.com/"],
        "register": "http://bower.myserver.com"
    }
}

This instructs bower to download packages from your server or the official one (the first has the priority), and to register the packages on your server only.

Automate everything !

Using grunt-bower-task you can create a build step that install all the dependencies required by your bower.json and copy all the files defined in "main" in a folder. If you use require.js you can also use grunt-bower-requirejs to build  the require.js configuration automatically.

I think this is enough to start working with bower. Enjoy!




Thursday, July 25, 2013

Server side modules with Architect

After my last notes on a client side modular architecture using require.js I'll talk about server side modules using Architect.

Modules

In my opinion every application (not only the bigger ones) needs to be divided into interdependent modules. This helps a lot when you deal with testing and enhancing your application.
In node.js applications, you will see and use this pattern very often:

//import a module
var Database = require('./db');

// initialize a module using some options 
var db = new Database('http://my.db.connection');

// initializing a module injecting a dependency 
var usermanager = require('./usermanager')(db);

But what if you want to use this module in other applications ? In this case you should make an npm independent package for each module, of course. And, in the main application you would initialize each package in the proper order. This is repeatitive and tedious furthermore, when the number of the packages starts to get bigger, initializing each module in the proper order can be an issue.

From modules to plugins

Architect uses a simple declarative syntax to explain the interconnections between packages. It also starts the system in the correct order initializing each package just once.
I recommend to read the documentation on github. It's very clear and detailed.

Express.js and Architect

I have set up a simple express.js/architect boilerplate to show how to use Architect to make an extensible express.js application.


I hope this will be useful ...

P.S.
A friend of mine suggest me to use a git subtree for each package. Nice idea!


Thursday, July 18, 2013

Client side modules with require.js


In this post I will show you how to use Require.js to split a project into simple and manageable modules.

UMD and AMD

I really didn't want to dive into differences of these 2 module systems but I think it's very important to realize of what module system we are talking about.

UMD is the module system used by node.js. You define modules doing this (foobar.js):

module.exports = {
    foo: 'bar'
};

And load a module doing this:

var foobar = require('./foobar');
console.log(foobar.foo); // prints bar

You can use UMD in the browser using browserify.
If you usually work with Javascript in the browser you will notice two issues:
  • the exports object would be overwritten every time by differents modules
  • it uses a synchronous approach
As a matter of fact it cannot work in the browser without a build step (and this is the browserify's task).

AMD instead is designed from the ground up to work in the browser.

Saying this I am not advocating one of these systems. They are both very useful even though they use different approaches.

I started by explaining UMD because, unfortunately, both systems use a function called "require".
Now that you can't be fooled anymore by this let's go on.

What is an AMD module

An AMD module must be contained in a single file and is encapsulated by the global function define.
"define" takes two parameters: an array of dependencies and the actual module code.

//module3.js
define(['module1', 'module2'], function(module1, module2) {
    'use strict';

    var namespace = {};
    
    return namespace;
});

In this example I have defined a module called "module3". This module needs module1 and module2 to run.
The return value of define will be returned if another module requires module3.
module1 and module2 are resolved loading with AJAX module1.js and module2.js (both AMD modules).
The job of require.js is basically to resolve the dependency tree and to make sure that every module would be run just once.

A module has a pair of interesting properties:

  • It is loaded the first time is required by another module.
  • not a single variable is added to the global namespace. For this reason you can even use different versions of the same library if you need to

Bootstrap and configuration

After defining modules you will need to bootstrap your application (configure and load the first module). For doing this you will define in your page a script tag with require.js and the url of the bootstrap script (main.js).

<script data-main="/js/main" src="js/vendor/require.js"></script>

After loading require.js it will load the "main.js" bootstrap using ajax. From this point every script will be loaded asynchronously and the DOMContentLoaded event (the jquery ready event) will be fired independently from the script loading.

Main.js is made of 2 part. The first one is require.js configuration:

require.config({
  baseUrl: "js/",
  paths: {
    jquery: 'vendor/jquery-1.9.1',
    underscore: 'vendor/underscore',
    backbone: 'vendor/backbone',
  },
  shim: {
      underscore: {
          exports: "_"
      },
      backbone: {
          deps: ['underscore', 'jquery'],
          exports: 'Backbone'
      },
      'jquery.bootstrap': {
          deps: ['jquery'],
          exports: '$'
      }
    }
});

Here are the most important parameters:
baseUrl: this is the path used to resolve modules.. So if you require "module1" this will be loaded from "js/module1.js".

paths is very useful to define script that are in some other paths. When you require "jquery" will be loaded the script "js/vendor/jquery-1.9.1.js".

Shims

Until now I assumed that every script is an AMD module but this is not always true.
require.js shim option allows to wrap automatically a non AMD script into an AMD define.

"deps" are dependencies to be injected and "exports" is the value to be returned:

For example your backbone.js will become:

define(['underscore', 'jquery'], function (_, $){

..actual backbone.js source ...

return Backbone;

});

The trick works even when you have to add attributes to an existing object (like defining a jquery plugin):

define(['jquery'], function ($){

$.fn.myJqueryPlugin = function (){
};

return $;

});

This is the case of Twitter bootstrap.

require

The second part of main.js is the actual bootstrap. It loads and execute the first module.

require([
  // Load our app module and pass it to our definition function
  'app',
], function(App){
  // The "app" dependency is passed in as "App"
  App.start();
});
The app module will start a chain of loading that will load the entire application's scripts.

The useful text plugin

The text plugin allows you to add text files as dependencies and this is very useful for client side templating.
Just add "text" in the path config:

require.config({
  ...
  paths: {
     ...
    text: 'vendor/text'

and you can load your templates:

define(['underscore', "text!templates/template.html"], function (_, template_html){
    var template = _.template(template_html);
    ...
});

You can also write your own plugin as described here.

Script optimization

Using a lot of modules have an obvious drawback: the time spent to load these modules in the browser.
Require.js comes with a useful tool for optimizing scripts: r.js. It analizes, concatenates and minifies all the dependencies in one single script.

I warmly recommend to use a build step to make this operation automatically.

I use grunt and a grunt plugin to automate everything.

Installing grunt and the require.js optimizer plugin

Grunt is structured in 2 different modules: "grunt-cli" and "grunt" and a series of plugins. "grunt-cli" can be installed globally:

npm install -g grunt-cli
grunt and the plugins should be installed locally pinning a release version. This system allows to use different versions and plugins for each project.
npm install grunt --save-dev

npm install grunt-contrib-requirejs --save-dev

The save-dev option adds the modules in the package.json under the "devDependencies" key and using the latest release.
...
  "devDependencies": {
    "grunt": "~0.4.1",
    "grunt-contrib-requirejs": "~0.4.1"
  },
...

You can also do this manually and launch "npm install".

Grunt configuration

Grunt needs a configuration file called Gruntfile.js this is an example from a past project of mine:

module.exports = function(grunt) {

  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    requirejs: {
      compile: {
        options: {
          mainConfigFile: "static/js/main.js",
          baseUrl: "static/js",
          name: "main",
          paths: {
            'socketio': 'empty:',
            'backboneio': 'empty:'
          },
          out: "static/js/main-built.js"
        }
      }
    }
  });

  // Load the plugin that provides the "uglify" task.
  grunt.loadNpmTasks('grunt-contrib-requirejs');
  // Default task(s).
  grunt.registerTask('default', ['requirejs']);

};

In the "requirejs" configuration I have:
mainConfigFile: the path of the bootstrap script
baseUrl: the same as configured in the main.js configuration
name: the name of the main script
paths: in this section I added a pair of script to not be included in the optimized script. These 2 files are served directly by my node module. You can also do the same for script served through a CDN or an external service.
out: the output file

"registerTask" allows me to launch the whole optimization step with the "grunt" command (using no options).

At the end of the task I can load my new optimized script using:

<script data-main="/js/main-built" src="js/vendor/require.js"></script>
I think this is all. Stay tuned for the next!

Edit: If you liked this you'll probably be interested in this other blog post on require.js edge cases.