Again: exports and module.exports in JavaScript Modules

I reread my posts and realized that I haven't been clear as I'd like.

If you require() a module there are a couple simple rules:

1. you don't need to put the '.js' on the end of the file.
2. if you require('./my_module') the file must be in
   
   1. ./my_module.js or
   2. ./my_module/index.js
3. if you require('my_module') the file must be in
   
   1. ./node_modules/my_module.js or
   2. ./node_modules/my_module/index.js or
   3. or a node_modules directory toward root (.. or ../.. or ../../.. till /node_modules)

As for the module file itself, you must set the object to be returned from the require() call. This is the part that flummoxed me.

It is simple, in your module there is a module variable automagically. That module variable points to an object that is pre-populated with many properties. Just do a console.dir(module) in your empty module file to see it all.

The object that gets returned from the require() call is pre-allocated in the module.exports property. There is also a variable called exports that contains the very same object that is in module.exports. If you assign a new object to the exports variable, eg exports = {};, that new object will NOT be exported as it is not the same object that is in module.exports. Think of it as if there is a invisible exports = module.exports = {}; line at the beginning of your module. The exports variable is only there for your convenience and only the object pointed at by module.exports matters.

This is simple and I found no one who said it straight out. There was crap about exports being an "alias" for module.exports and other balderdash. If you want to create your own elaborate exported object just do this:

exports = module.exports = {
 ... my big-ass exported object ...
};

CommonJS exports spec ambiguity

I have just been reading the spec for CommonJS. In the 1.0 Spec it refers to the exports variable and object interchangeably.

2. In a module, there is a free variable called "exports", that is an object that the module may add its API to as it executes.
3. modules must use the "exports" object as the only means of exporting.

There is no mention of "module" or "module.exports".

I've also realized I am a blithering idiot. I put console.dir(module); in an empty module and I saw that it was populated with a great many things. Most pertinently, module has a property named exports already initialized to an empty object. If you add something to module.exports like module.exports.foo = 'bar'; then console.dir(exports); will output { foo : 'bar' }. It is sad how much my coding skills have atrophied the last few years. I used to be rather sharp ... oh well.

Nodejs: module.exports and exports

I have just had a rough time learning how to create my own module. The problem is that I can't move on just knowing How to get something to work, I need to know Why. It is some form of programmer paralysis; worse than writers block. I needed to know how/why exports worked in Nodejs modules.

Requiring modules is simple

var m = require('./my_module');
m.myFunc(foo, bar);

The file needs to be "./my_module.js", with the following exports line:

var myFunc = function () { ... };
module.exports.myFunc = myFunc;

The Why is the hard part. You can't do exports = {}, but you can do module.exports = {}. Turns out module is the "global" namespace in an require() file. exports is some sort of "alias" for module.exports (I haven't figured that out). What this means is that exports.myFunc = myFunc; is OK, but exports = {myFunc: myFunc} is NOT OK, BUT module.exports = {myFunc: myFunc} is OK. Basically, I've decided not to use this pseudo-whatchamacalit "alias" type thing exports. I'll just use module.exports. Additionally, it is probably best to use the form:

module.exports.myFunc = function () {
...
};

for all exported functions inside ./my_module.js

Testing GitHub's embeded gist

Bugger! No line numbers. I'll look into that.

Syncronous coding style with EventMachine and Fibers

I did this a while ago. I am not really happy with it. As I've said before. Event driven code doesn't bother me. Here is an example.

require 'fiber'
require 'eventmachine'

class AsyncIO < EM::Connection
  include EM::P::LineText2
  def initialize df
    @df = df
  end
  def receive_line line
    @df.succeed line
  end
end

def get_line
  f = Fiber.current
  df = EM::DefaultDeferrable.new
  df.callback { |line| f.resume(line) }
  EM.connect '127.0.0.1', 3000, AsyncIO, df
  return Fiber.yield
end

EM.run {
  Fiber.new {
    line = get_line
    puts line
    EM.stop
  }.resume
}

The truth is you should look at EM::Sychrony. Ilya Grigorik explains in he own Blog, how Fibers can help you write synchronous code while keeping the goodness of AsyncIO (ala EventMachine).

I don't find event driven code to be "callback hell" like Ilya Grigorik does. I find evented code easier to read than synchronous code sometimes. In Synchronous code you have to check every time if it failed and why. That is

if (rc < 0)

with

switch (errno) case EINTR:

code after every network call. In event driven code that stuff is dealt with in the event-loop and you just have to define the error callback once.

Ah well just use Node.js . If JavaScript event driven programming is "to hard" for your delicate sensibilities then explain to me how Millions of Web-monkeys pull it off everyday in the browser.

Morans! I am surrounded by Morans!

I posted a response on a blog recently. The fellow claimed, quite explicitly, that "Fibers and EventMachine were a response to Ruby's poor Threading performance". I responded with Fibers and EventMachine were not made in "response" to Ruby's Thread implementation. I am strangely polite in online discussions, where in person I'd have ripped this guy a new asshole.

I explained that Threads exist because n = read(fd, buf, len) blocks. Threads allow execute work to continue in parallel to blocked IO. That he had it backwards, event driven IO, like EventMachine, exist so a user-space program can continue to execute work while the IO finishes in the Kernel. While user-space programmers don't explicitly invoke the switching of one thread to another, context switching is not costless. In fact the cost of context switching is many hundreds of CPU instructions.

Relatedly, Fibers are cheap where Threads are expensive. Fibers switch into a parallel call stack, much like Threads, with the equivalent of a register switch and a long jump. Additionally, the ability to return results from a Fiber while preserving the Fiber's call stack state allow for remarkable new behaviors to be constructed that can not be done with out them.

It seems these Ruby coders one of two types: One, HTML monkeys that have learned how to code, or Two Java refugees that got tired of Bondage-and-Discipline style of programming. Both of these kind of MORANS, don't understand that Ruby is a remarkable Programming language with features not available in other languages. Neither Perl nor Java, have the concept that every variable is an object. Nor do either language have Fibers/coroutines. These features change the very nature of the code you write.

P.S.
I plan to post some code to show synchronous read/write IO calls while doing all the actual IO inside EventMachine. So you get the best of both worlds ala synchronous IO with out blocking. Stay tuned; same Bat Time same Bat Channel...

Git and RSpec

I just watched a talk by Linus Torvalds at Google about Git. One of the questions touched on one of Linus' motivations for Git. That is Merging. I want to talk about how Git merging dove-tails nicely with Ruby's testing tool RSpec.

Some source control managers (SCMs) make a big deal about "branching" being cheap. Linus points out that branching is not the problem, merging is the problem people deal with. Git makes merging easy by removing the number of conflicts the developer has to deal with. It does this two ways. First is algorithmic and the second is how Git changes your workflow.

Git allows for, and relies on, three way merging. CVS/SVN by contrast only does two way. Three way merging diffs the original file and each of the two conflicting versions of that file. Git can do so because it stores the whole files, not just a long string of diffs (ala CVS/SVN which I'll just call SVN from now on). Three way merges allow the merge algorithm to look at the context of the text being diffed. SVN can only look at line 245 and see it changed. Git can notice that the line didn't change, it mearly moved. Say you and the conflicting file inserted a text before the same line 245. So there is no conflict just code motion in the file. Three way diffs see this, two way diffs can't.

The second way Git deals with merge conflicts is not as obvious; it stems from the work flow. Specifically, branching is inherent in Git's design and merging is easier and better. So you branch often and merge often. More to the point you start your branch from the central "master" branch for your feature add; change & commit a bunch of times to that small branch; then pull or push your change back into "master". By repeating this branch-change-merge-push cycle early and often you have less opportunity for merge conflicts. Put another way, your branch exists for less time and syncs with the central master more often, resulting in less opportunity for two people to introduce genuine conflicts.

So alot of talk is generated about Git working in a fully distributed way. However, that ignores that the Git, encourages syncing up those distributed merges early and often.

So imagine you replace SVN with Git. First you don't have to do it to your whole tree at once. You can do it by subsystem. Also there is Git-SVN gateway tools to help do this. You can still have the central tree ala SVN, but you just branch off it and merge back more often. In SVN you are usually doing this anyways. Each SVN commits end up being after the feature is completed; much like the final merge of a Git branch, but without all the Git commits in the mean time. Or you create a longer lived development branch and merge that back as a monster merge.

I've explored other SVN branching/merging workflows but all they do push the pain to a different part of the process. A smart colleague of mine called it "squeezing the balloon". What you need to do is keep the master branch stable. That can be achieved by branching and merging smaller and faster. The SCM (or VCS if you prefer) makes all the difference.

What also helps is having a test suite you can rely on. First the test suite has to be aimed at the internal API level. Second, it has to be easy and fast to run. This is where the choice of Test framework dove tails with the SCM choice. "Early and Often" is the catch phrase for both.

Testing the API's "contract" is where you should aim your testing framework. As a side note: you are forced to state what that "contract" is which is a level of internal documentation that often gets forgotten. The contract is what each function takes as input and what the results should be; especially the edge cases. Edge cases are given a function "sum" adds all the elements of an array, what if the function is given an empty array or a nil as input. So that is the trivial case. Then there is the out-of-bounds case, like each element of an IP address (dot-quad) is between 0(inclusive) and 256(exclusive). Correctness should be tested if you can. Most of the time, correctness can't be tested with out recreating the logic of the function or testing pre-canned input and results. But that leads of who-tests-the-tester (my favorite quote: "Qui custodiet ipsos custodes?")

Another variation of this "contract" is how it applied to methods of a object. Methods have input and return results, but they may alter the objects state. Input and result have to be tested as above, edge case and out-of-bounds case. But internal state? One internal state to test is internal consistency. Again there are edge cases of initialized(trivial states) and known inconsistent states.

RSpec is a good ruby oriented tool-set built explicitly to test the contracts in your APIs. What are other API test frameworks for other languages is left as an exercise for the reader :)