In response to some solutions on the board that failed for various reasons:

1. Allows multiple elements to be inserted (no other version I found did this, usually because they worked by appending and using .last())
2. Maintains events and data (solved by .clone(true) on .children())
3. Works with 0-children containers (by using children.before())
4. Allows out-of-bounds and negative indexes (handles just like array.splice)

This uses the native .splice() method so is eminently predictable.

$.fn.insertAt = function(elements, index) {
    var children = this.children().clone(true);
    var array = $.makeArray(children);
    array.splice(index, 0, elements);
    this.empty().append(array);
    return this;
}

POC: http://jsfiddle.net/moagrius/Qvvau/2/