Iced Blog

beefy workflow with requirejs

Sometimes you want to work on your AMD code as a single bundle; the same way it will be deployed in production, with automatic bundling and live reload. Thanks to Chris Dickinson's recent work on the beefy development server, it's now easier to use r.js as a custom bundler.

Let’s get it configured.

What is beefy?

beefy is a static web server with built-in JavaScript bundling.

It was made to work with browserify by default, but if you use the --bundler command line argument, you can specify r.js as your bundler of choice.

We are going to use this to run r.js each time our main.js file is requested.

Get beefy by using the command:

npm install -g beefy

What is r.js?

r.js is a command line utility designed for running AMD-based projects in node.js, Rhino, and xpcshell. It also includes an optimizer for combining your AMD files into a bundle.

By using the optimizer functionality of r.js, in combination with beefy, we can bundle our JavaScript on each page refresh.

We need to install r.js into our project using the command:

npm install requirejs

If you have a package.json and want to add it to your devDependencies, add --save-dev to the command:

npm install --save-dev requirejs

Minimum Viable Config

There are seven options we need to set in our r.js config file that will allow us to use it with beefy (actually, two are just nice to have). They are baseUrl, name, insertRequire, optimize, useSourceUrl, out and logLevel and each one is explained below.

These options will be set in a file named config.js, which will be passed to the beefy command.

You can find an example project at https://github.com/iceddev/beefy-requirejs-example

baseUrl

As with every Require.js project, you are going to want to set your baseUrl.

This could be any place you want to begin resolving. Typically, it will be your vendor or application directory. Since we are doing a simple example, let's just use our current directory and we will put our JS files at the root of our project.

// config.js
({
  baseUrl: './'
})

name

To specify the module (and all of its dependencies) we want to optimize, we use the name option. This option indicates an entry point for r.js to begin resolving dependencies.

We are going to name our entry module main.js, an AMD package convention, and it will be in the root of our project.

// config.js
({
  baseUrl: './',
  name: 'main' // drop the .js, as AMD moduleId resolution appends it
})

main.js

Create a file in the root of your project named main.js.

This file will be the entry point for your applicaton, and will be wrapped in a define function call.

// main.js
define([
  './another-module'
], function(another){

  console.log('inside main.js');

  console.log('another-module dependency:', another);

});

another-module.js

Create another file in the root of your project, but name this one another-module.js.

This file is a dependency of your main.js module and is just used to demonstrate that we are, in fact, bundling all the dependencies together.

// another-module.js
define(function(){

  console.log('inside another-module.js');

  return {
    something: 'yup, another module'
  };

});

insertRequire

We can specify the insertRequire option, to insert a require function call at the bottom of our bundle, which will initialize your application.

Note: This isn't needed if your data-main filename is the same as the module entrypoint name because Require.js will do an implicit require. We will add it here because it doesn't hurt anything and will help in situations like naming the files differently or using something like almond.

// config.js
({
  baseUrl: './',
  name: 'main',
  insertRequire: ['main'] // will add `require(['main'])` to the end of your bundle
})

optimize

During development, you probably want to disable optimization/uglification, so you will be able to debug your bundle and builds will happen quicker. This is done by setting the optimize option to 'none'.

// config.js
({
  baseUrl: './',
  name: 'main',
  insertRequire: ['main'],
  optimize: 'none'
})

useSourceUrl - Source Maps

When using the optimize: 'none' option, we can get source maps, using //# sourceURL= and an eval call.

Setting the useSourceUrl option to true will auto insert these for you, but it should be disabled when bundling for production.

// config.js
({
  baseUrl: './',
  name: 'main',
  insertRequire: ['main'],
  optimize: 'none',
  useSourceUrl: true
})

out

The typical way r.js is used is to output a file, determined by providing a filename string as the out option.

beefy doesn't operate on files, and instead expects to receive data on process.stdout.

The out option can also take a function that will receive the output text as its only parameter.

We are going to leverage the out function to redirect the r.js output to process.stdout.

// config.js
({
  baseUrl: './',
  name: 'main',
  insertRequire: ['main'],
  optimize: 'none',
  useSourceUrl: true,
  out: console.log // console.log outputs to process.stdout and is tightly bound in node
})

logLevel

By default, r.js logs info about the build process. This gets intercepted by beefy on process.stdout and is added to the output served.

r.js provides a logLevel option that can be used to disable logging. Log level 3 is the level that logs only errors.

// config.js
({
  baseUrl: './',
  name: 'main',
  insertRequire: ['main'],
  optimize: 'none',
  useSourceUrl: true,
  out: console.log,
  logLevel: 3
})

index.html

The last thing we need is an index.html file that includes Require.js. If an index.html file doesn't exist, beefy serves up a default page that just injects a script tag for your entrypoint file. This won't work with the workflow outlined above because we assume the require machinery will be available.

Create an index.html file in the root of your project that contains:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Beefy + Require.js</title>
</head>
<body>
  <script src="/node_modules/requirejs/require.js" data-main="main" type="text/javascript"></script>
</body>
</html>

Run beefy

The beefy command takes a filename as the first argument, or an input filename/output filename combination in the form of input-filename.js:output-filename.js. If you don't specify an input filename but specify an output filename, in the format :output-filename.js, beefy won't pass a filename to the bundler, but it will still make the result of the bundler command available as output-filename.js.

r.js assumes it is supposed to run a file if one is passed as the first argument to the command, and skips the optimization tool. To avoid this, we will specify the first beefy argument as :main.js.

Next, we want to reference the r.js compiler as the bundler: --bundler ./node_modules/.bin/r.js

Finally, r.js expects the -o config.js argument to start the optimize tool with the config.js file. Anything after the -- argument to beefy is passed directly to the bundler command.

Putting it all together, the command looks like:

beefy :main.js --bundler ./node_modules/.bin/r.js -- -o config.js

Accessing your bundle

Open your browser and go to the location that the beefy command said it is listening on, e.g. listening on http://localhost:9966/.

You should see logging in your console if your modules loaded correctly. You should also be able to view the individual files in the sources pane, probably under (no domain) since we were using the useSourceUrl option.

What else can we do?

This configuration will allow you easily to swap dependencies with a single line change.

// config.js
({
  // rest of the config
  paths: {
    underscore: 'node_modules/underscore/underscore'
  }
})

Can be changed to:

// config.js
({
  // rest of the config
  paths: {
    underscore: 'node_modules/lodash/dist/lodash'
  }
})

And your underscore references will resolve to Lo-Dash on the next refresh of your page.

Watch out for part two of this article for some advanced techniques and any other interesting stuff I find related to AMD and beefy.

-Blaine

View Comments

This past weekend (July 27, 2013) was International Nodebots Day and I decided to attend Portland's event.

The event was excellent and I would like to give props to all the organizers, Tracy, Troy, Adron, Carter, Jason, Adam & anyone else I might have forgotten, who were able to bring people of all skill levels and interests together to control hardware with JavaScript.

Some projects I had interest in working on involved using different input devices to control robots with a small JS layer in between.

The hardware I brought to work with was LeapMotion, Google Glass, and Shieldbots. I was also hoping someone would bring a NodeCopter.

The goal was to use the input devices to control the bots, all using JavaScript.

Flying a Drone with Glass #1

Photo by Alice Goldfuss

Leap + Bots

I got started a day early with Troy and Nick while hacking at the New Relic offices.

The first thing we did was read the input data from LeapMotion's WebSocket connection and use it to drive a ShieldBot around using johnny-five, shieldbot.js, and leapjs.

var five = require('johnny-five');
var Shieldbot = require('shieldbot');
var leap = require('leapjs');
var board = new five.Board();

board.on('ready', function(){

  var shieldbot = new Shieldbot();
  var prevZ;

  leap.loop(function(frame){
    if(frame.hands.length){
      var z = frame.hands[0].palmPosition[2];
      var velZ = frame.hands[0].palmVelocity[2];

      if(prevZ == null){
        prevZ = z;
      }

      var direction;
      if(z < prevZ && Math.abs(velZ) > 50){
        direction = 'forward';
        shieldbot.forward(127, 30);
      } else if(z > prevZ && Math.abs(velZ) > 50){
        direction = 'backward';
        shieldbot.reverse(127, 30);
      } else {
        direction = 'still';
        shieldbot.stop();
      }

      prevZ = z;
    } else {
      shieldbot.stop();
    }
  });

});

We used the Leap's z-axis velocity tracking of a user's palm to drive the bot forward and backwards.

During Nodebots Day, I was able to add left & right stearing using palm x-axis rotation.

// outside leap.loop
var prevX;

// inside leap.loop
var velX = frame.hands[0].palmVelocity[0];
var x = frame.hands[0].palmNormal[0];

if(direction === 'still'){
  if(x > prevX && Math.abs(velX) > 50){
    shieldbot.turnRight(127, 30);
  } else if(x < prevX  && Math.abs(velX) > 50){
    shieldbot.turnLeft(127, 30);
  } else {
    shieldbot.stop();
  }
}

prevX = x;

Other plans included using the finders to stop the bot running into things or off tables, but I was unable to implement these because I quickly moved on to my next input device, Google Glass.

Glass + Bots

For his JSConf talk, Luis created a sideloadable app, Face, for Glass that broadcast the sensor data over WebSockets. With this, I was able to get pitch and roll, which were both easily translated to the logic of the LeapMotion/ShieldBot combo.

var five = require('johnny-five');
var Shieldbot = require('shieldbot');

var io = require('socket.io-client');
var socket = io.connect('http://localhost:11097');
socket.on('connect', function(){
  socket.emit('joinChannel', CHAN_ID_FROM_FACE);
});

var board = new five.Board();

board.on('ready', function(){

  var shieldbot = new Shieldbot();
  var prevZ;
  var prevX;

  socket.on('broadcast', function(message){
    if(message.sensors){
      var z = message.sensors.pitch;
      var x = message.sensors.roll;

      if(prevZ == null){
        prevZ = z;
      }

      var direction;
      if(z > prevZ && Math.abs(z - prevZ) > 1){
        direction = 'forward';
        shieldbot.forward(127, 30);
      } else if(z < prevZ && Math.abs(z - prevZ) > 1){
        direction = 'backward';
        shieldbot.reverse(127, 30);
      } else {
        direction = 'still';
        shieldbot.stop();
      }

      if(direction === 'still'){
        if(x > prevX && Math.abs(x - prevX) > 1){
          shieldbot.turnRight(127, 30);
        } else if(x < prevX  && Math.abs(x - prevX) > 1){
          shieldbot.turnLeft(127, 30);
        } else {
          shieldbot.stop();
        }
      }

      prevZ = z;
      prevX = x;
    } else {
      shieldbot.stop();
    }
  });

});

After replacing some variables and reworking my velocity check, I was able to drive the bot forward and backward, matching the direction I tilted my head. Rolling my head left and right also turned the bot.

This demo wasn't ideal because I couldn't see where the bot was while controlling it. I needed something that could be about eye height, so I could see its movement while controlling it. Luckily, Carter had brought a NodeCopter!!

Glass + NodeCopter

If it was so easy to switch the LeapMotion logic to use the Glass data, I figured it would be just as easy to swap the ShieldBot logic for the NodeCopter's. After discussing the idea with a few people, it was suggested I use the Voxel Drone to test my logic before applying it to a physical drone. After a very frustrating experience trying to get Voxel Drone, which seems to have been abandoned, to actually work, I was able to wire up my Glass/NodeCopter logic and added code for the drone to take off with the A button and land with the B button displayed in Glass.

var io = require('socket.io-client');
var socket = io.connect('http://localhost:11097');
socket.on('connect', function(){
  socket.emit('joinChannel', CHAN_ID_FROM_FACE);
});

var prevZ;
var prevX;

socket.on('broadcast', function(message){

  if(message.event){
    if(message.event === 'A'){
      drone.takeoff();
    }
    if(message.event === 'B'){
      drone.land();
    }
    return;
  }
  if(message.sensors){
    var z = message.sensors.pitch;
    var x = message.sensors.roll;

    if(prevZ == null){
      prevZ = z;
    }

    var direction;
    if(z > prevZ && Math.abs(z - prevZ) > 1){
      direction = 'forward';
      drone.front(1);
    } else if(z < prevZ && Math.abs(z - prevZ) > 1){
      direction = 'backward';
      drone.back(1);
    } else {
      direction = 'still';
      drone.stop();
    }

    if(direction === 'still'){
      if(x > prevX && Math.abs(x - prevX) > 1){
        drone.right(1);
      } else if(x < prevX  && Math.abs(x - prevX) > 1){
        drone.left(1);
      } else {
        drone.stop();
      }
    }

    prevZ = z;
    prevX = x;
  } else {
    drone.stop();
  }

});

I loaded up Voxel Drone and was able to fly a WebGL drone in my browser.

All I needed to do was swap the voxel-drone specific code for:

var arDrone = require('ar-drone');
var drone  = arDrone.createClient({
  ip: '192.168.1.10'
});

Demos

Flying a Drone with Glass #2

Photo by Chris Hansen

Turns out that I was driving the drone at full speed in each direction I tilted my head. I should have had the speed at about 0.3 instead of 1. Lesson learned. I would have also liked to add the ability to rotate the drone left and right based on the Glass' azimuth value, but I guess that will have to be in the future.

Wrapping Up

With JavaScript, we are able to write a relatively small amount of code that can serve as a basis for swappable hardware components.

I was able to start with a LeapMotion and Shieldbot and end up using Google Glass and a NodeCopter with quite similar code.

After such an amazing Nodebots Day, I have all sorts of ideas swimming around in my head for taking these types of interactions further!

-Blaine

View Comments

node js in chrome

This:

<script type="text/javascript" src="node.js"></script>

Allows you to do this in a Chrome App:

var http = require('http');
http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Hello World\n');
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');

This is actually Node.js code running the http listener on Chrome's Javascript VM!

While we like the Browserver project, it uses a reverse proxy and pushes requests down to the client via a websocket. Node-chromify took a different approach, and has no external server dependency.

You can get the source to the demo app, and a build of node.js (the script, not the whole node project) here: https://github.com/iceddev/node-chromify

So, how did we do this?

node-chromify is built on top of browserify, which allows node modules to be converted into a format that can be run in a browser. Browsers can't normally created TCP listeners, but Chrome Apps can in Canary with the experimental extension APIs flag enabled.

Google's Paul Kinlan created a net-browserify module which wraps the Chrome API calls to the node.js "net" package. This gets us quite a bit further. We've patched a few things on this to help us out.

We've created an http-chromify module which is based on Node.js' "http" module. The Node.js version uses native C++ code to parse the HTTP messages between client and server. We replaced that with the pure JS http-parser-js module.

There were also couple of other pure JS modules that weren't browserified yet, namely freelist and string_decoder. We created repos for those so that browserify could pull them in.

Finally we used the excellent Grunt.js to build and package this all up into node-chromify.

How do you run it?

You'll need Chrome Canary to run the demo. Canary has recently allowed developers the ability to open TCP socket listeners in apps and extensions.

Also, this requires enabling "Experimental Extension APIs" from chrome://flags

Then, in chrome://extensions use the "Load unpacked extension..." to point to the examples/plugin folder in the node-chromify source.

We're accepting pull request to node-chromify, please help us fix this :)

Why did we do this?

"any application that can be written in JavaScript, will eventually be written in JavaScript."

It's the law. Obey the law.

View Comments

At Iced Dev, our main business is consulting for Node.js and building great web applications using it with HTML5 technologies. Our favorite hosting platform is Nodester (@Nodester) because it is the only Open-Source PaaS for Node.js and maintained by a great group of developers. Recently, Nodester redesigned their website and made it extremely easy to use their API using apigee’s API Explorer. This is very helpful for people using Windows because curl is no longer required to request an invite, but Window’s users may still have trouble using Nodester because Git is used to push your project to the Nodester platform. For our Node.js development, we typically use Cloud9 IDE because it is very easy to debug our Node.js apps before we deploy them to production. Unfortunately, Nodester is not a platform that is supported by Cloud9’s deploy feature (We have no idea why this is!!! Cloud9, please make this happen!). Thankfully, Cloud9 uses Git and makes all the commands available to the user. Using the new Nodester API Explorer and Cloud9’s Git implementation, I am going to walk you through publishing your first Node.js app to Nodester from any every operating system with a web browser.

Set up your Cloud9 account.

  1. Go to http://c9.io/
  2. Choose a username
  3. Enter a valid email (they send you an activation email)
  4. Re-enter your email for confirmation
  5. Click the “Sign Me Up” button
  6. Check your email for your confirmation email
  7. Follow the activation link in your email
  8. Choose your password (following any password requirements C9 has)
  9. Re-type the password for confirmation
  10. Click Activate
  11. You will be redirected to your Cloud9 Dashboard (Keep this tab open for now)

Request a Nodester Registration Coupon.

  1. Open (in a new tab) http://nodester.com/
  2. Once on Nodester.com, Request your Registration Coupon by clicking the blue button labeled “Request Registration Coupon”
  3. Enter your email address in the box labeled “Email” and click Send
  4. Wait patiently to receive your Nodester Coupon (many more are being sent out daily)
  5. Once you receive your Nodester Coupon, you will be able to sign up for an account at http://nodester.com/api.html\#explorer

Set up your Nodester account

  1. Expand the “User” option
  2. Click “CreateUser”
  3. The API Explorer will populate the URL
  4. To the right of the URL input, click the little gear button (Apigee Gear Icon)
  5. Clicking the button will bring up the Advanced Options dialogue box
  6. Enter the Nodester Coupon received in the email in the “value” box next to “coupon”
  7. Enter your email in the “value” box next to “email”
  8. Enter a password in the “value” box next to “password”
  9. VERY IMPORTANT:The RSA Key you enter must be the RSA Key provided by Cloud9 IDE

    • On the Cloud9 Dashboard, click “Show Your SSH Key” on the right side, in the “Account Settings” list
    • Click “Copy” to copy your RSA Key to the clipboard
    • Paste that RSA Key into the Nodester “Advanced Options” dialogue in the “value” box next to “rsakey”
  10. Enter a username in the “value” box next to “user”

  11. Leave all the other options as defaults
  12. Click “OK” to save the options and dismiss the dialogue
  13. Click the “POST” button to submit your API call

    • Your Response should look something like:

      HTTP/1.1 200 OK
      Content-Length:
      20
      Content-Type:
      application/json; charset=utf-8
      Connection:
      close
      X-Powered-By:
      Express{
      "status": "success"
      }
    • If your response does not have the “status” of “success”, you have to repeat the above steps again (most likely because something was mistyped, that username is taken, or something else funky happened).

  14. Now that you have a Nodester account, it’s time to create your first Nodester app!!!!

Create a Nodester App

There are 2 options for creating a Nodester app with the API, I will walk you through the Nodester Admin Panel (The option I like better) and then I will walk you through using the Nodester API Explorer.

Nodester Admin Panel

  1. Since you have a Nodester account, you can access the Admin Panel by entering your username and password at the top of the page.

    • As a new user, your will get an error message on the first page you see in the Admin Panel, but that is only because it is looking for your apps and you don’t have any yet.
  2. Click “Create New App” in the menu on the left

  3. Enter the name of the app you wish to create (This is also the subdomain that your app will be accessed with, e.g. appname.nodester.com)
  4. Enter the file name that you wish to be your startup file (This file will most likely be where your Node.js app is configured to listen on your specific port)

    • Personal preference:I like to use server.js as my start file because I have encountered issues using app.js as the start file with Nodester.
  5. Once your Nodester app is created, you will see something similar to: [Nodester Admin Panel Clipping](Nodester Admin Panel Clipping)

    • The port is the port that your app listens on (this will need to be hardcoded into your app) and the app-status is false because there is no code inside the app yet.

Nodester API Explorer

  1. If you would rather use the API Explorer, the steps are below:
  2. Expand the “App” option
  3. Click “CreateApp”
  4. On the left side of the URL input box, click the lock icon (Apigee
Padlock Icon) and select “Basic HTTP”
  5. Enter your Nodester username and password in the dialogue that opens
  6. Open the “Advanced Options” dialogue
  7. Fill in the values for “appname” and “start”
  8. Click “OK”
  9. Click “POST”
  10. The response you should get looks like:

    HTTP/1.1 200 OK
    Content-Length:
    182
    Content-Type:
    application/json
    Connection:
    close
    X-Powered-By:
    Express
    
    {
      "status": "success",
      "port": 13092,
      "gitrepo": "git@nodester.com:/node/git/nodester-deploy/4939-840ce8a201f03c7d27852a5bde15b55e.git",
      "start": "server.js",
      "running": false,
      "pid": "unknown"
    }
  11. Let’s get this app into Cloud9, so we can add some code and get it running!

Import Nodester App into Cloud9

  1. Click “Info” and copy the “gitrepo” value from the dialogue box that opens
  2. In your Cloud9 Dashboard, click the plus button that it points to while saying “Create a Project Here”
  3. Choose the “Clone from url” option and paste your “gitrepo” value from Nodester into the box, then click the “Checkout” button
  4. Cloud9 will perform the git clone operation and (once finished) you will be able to open the project within Cloud9 by selecting the project and clicking “Start Editing”
  5. Now that you have your project cloned into Cloud9, let’s add some code!

Add “Hello World” Code

  1. When your project opens, we need to create the start file specified when creating your Nodester app (I used server.js) by clicking the plus icon to open an untitled document.
  2. For this tutorial, we will just get this app running the Nodejs.org “Hello World” code with some small tweaks to get it to run on Nodester and still debug on Cloud9

    • Cloud9 requires you to use process.env.C9_PORTas your port for Node’s .listen()method and Nodester requires you to use a port specified to your app (They will have this environmental variable working soon), so I use this modified code for my “Hello World” app:

      var port = process.env.C9_PORT || 13087; /* Change 13087 to the port specified for your app */
      
      var http = require('http');
      http.createServer(function (req, res) {
        res.writeHead(200, {'Content-Type': 'text/plain'});
        res.end('Hello World\n');
      }).listen(port);
  3. Now, you want to save the code in the untitled document as the filename you specified as your start file when creating your Nodester app.

  4. Once saved, clicking the “Run” button will bring up a dialogue box for a run configuration, where you can name your configuration and specify which file it is to run as the start file (server.js for me). You can then click “Run” on that dialogue box.
  5. In the “Output” tab of your Cloud9 console, you will see a tip with a clickable link, when click, will launch the webpage for your app and you should see “Hello World” printed to the screen.
  6. Now that you have your app working in Cloud9, let’s get it onto Nodester!

Pushing your App to Nodester

  1. Pushing your app to Nodester requires a little knowledge of Git (even though you don’t install it on your local machine, Cloud9 is still using it behind the scenes).
  2. In the Cloud9 Command Line (at the bottom of the screen), we around going to want to issue the command (either type or copy-paste):

    git add .

    to add all files to the Git repo.

  3. Next, we want to issue the command:

    git commit -m 'initial commit'

    to make our initial commit to the Git repo.

  4. The last command we need to issue is

    git push origin master

    to push our changes to the Nodester Git repo.

  5. Nodester will report on the progress of the push and the restarting of your app with the committed changes

  6. If everything went as expected, your app will now be online!

Test Your Nodester App

  1. Direct your web browser to YOUR_APPNAME.nodester.com (my test app was c9deploy1.nodester.com) to make sure your app is running.
  2. You can stop and start your app from the Nodester Admin Panel if something isn’t working correctly.
  3. After all of this is setup/configured, the only 3 commands you need to remember are the Git commands to add, commit, and push your changes to your Nodester apps!!

That’s All Folks!

I hope this post helps some people (especially ones developing on Windows or ChromeOS) with the difficult task of getting your apps hosted on the Nodester platform using the equally awesome Cloud9 IDE.

View Comments