I’ve just released a simple chef cookbook that will install nodejs from source. You can check it out directly from github or download it from the opscode cookbook site. Let me know what you think if you find it useful.

Ignacio Nin and I (mostly Ignacio) have worked together to create tcprstat[1], a new tool that times TCP requests and prints out statistics on them. The output looks somewhat like vmstat or iostat, but we’ve chosen the statistics carefully so you can compute meaningful things about your TCP traffic.

What is this good for? In a nutshell, it is a lightweight way to measure response times on a server such as a database, memcached, Apache, and so on. You can use this information for historical metrics, capacity planning, troubleshooting, and monitoring to name just a few.

The tcprstat tool itself is a means of gathering raw statistics, which are suitable for storing and manipulating with other programs and scripts. By default, tcprstat works just like vmstat: it runs once, prints out a line, and exits. You’ll probably want to tell it to run forever, and continue to print out more lines. Each line contains a timestamp and information about the response time of the requests within that time period. Here “response time” means, for a given TCP connection, the time elapsed from the last inbound packet until the first outbound packet. For many simple protocols such as HTTP and MySQL, this is the moral equivalent of a query’s response time.

The statistics we chose to output by default are the count, median, average, min, max, and standard deviation of the response times, in microseconds. These are repeated for the 95th and 99th percentiles as well. Other metrics are also available. Here’s a sample:

[root@server] # tcprstat -p 3306 -n 0 -t 1
timestamp	count	max	min	avg	med	stddev	95_max	95_avg	95_std	99_max	99_avg	99_std
1276827985	1341	24556	23	149	59	767	310	91	69	1030	107	112
1276827986	1329	12098	28	134	63	461	299	91	65	667	104	93
1276827987	1180	13277	22	202	93	873	439	103	79	1523	131	169
1276827988	1441	15878	27	180	139	672	427	116	79	1045	136	128
1276827989	1432	157198	26	272	138	4165	405	115	80	1092	134	123
1276827990	1835	25198	26	183	124	734	448	115	85	1141	137	141
1276827991	1242	6949	29	129	114	301	233	98	61	686	109	84
1276827992	1480	284181	25	442	127	7432	701	128	114	4157	173	293
1276827993	1448	9339	22	161	88	425	392	104	80	1280	126	140

tcprstat uses libpcap to capture traffic. It’s a threaded application that does the minimum possible work and uses efficient data structures. Your feedback on the kernel/userland exchange overhead caused by the packet sniffing would be very appreciated — libpcap allows the user to tune this exchange, so if you have suggestions on how to improve it, that’s great.

We build statically linked binaries with the preferred version of libpcap, which means there are no dependencies. You can just run the tool. In the future, packages in the Percona repositories will provide another means for rapid installation via yum and apt.

tcprstat is beta software. Several C/C++ experts reviewed its code and gave it a thumbs-up, so many eyes have been on the code. We’ve performed tests on servers with high loads and observed minimal resource consumption. I personally have been running it for many weeks on some production servers without stopping it and have seen no problems, so I am pretty sure it has no memory leaks or other problems. Nevertheless, it’s a first prototype release, and we want much more testing. We might also change the functionality; as we build tools around it, we discover new things that might be useful. When we’re happy with it and you’re happy with it, we’ll take the Beta label away and make it GA.

The tcprstat user’s manual and links to downloads are on the Percona wiki. Commercial support and services are provided by Percona. Bug reports, feature requests, etc should go to the Launchpad project linked from the user’s manual. General discussion is welcome on the Google Group also linked from the user’s manual.

[1] Historical note: we initially called this tool rtime, but did not publicize it. However, some of you might have heard of “rtime” before. This is the same tool.

---

Entry posted by Baron Schwartz | 11 comments

Add to: | | | |

So earlier today the nice folks over at twitter figured it was a good time to change all the authentication to oauth ... they might have announced it all over the place .. but it never catched my attention

The onlything that did catch my attention was that after not having ued pidgin for about 2 weeks I didn't have access to twitter anymore.

I`m using the purple-microblog plugin and the default version of that plugin in Fedora 12 wasn't really up2date. The plugin supports OAuth as of 3.0 which was released ages ago.

The version in fedora-updates-testing however was already recent enough ..

So enabling that repo and running
yum  update  purple-microblog
quickly solved my proble .. till I disabled twitter in my pidgin again as there was way to much talk about some weird fruit ...

-->

Thanks to the around 175 developers who came to Yahoo! recently for our monthly Hadoop User Group meeting. The energy in the packed room was phenomenal, and conversations continued long after the formal sessions.

Hundreds of Hadoop Fans Flock to Yahoo! for the Hadoop User Group

The event started with Arun Murthy from Yahoo! describing the best practices for developing MapReduce applications. Arun introduced the concept of a Grid Pattern which, similar to Design Pattern, represents a general reusable solution for applications running on the Grid. Finally, Arun talked about the anti-patterns of applications running on the Apache Hadoop clusters.

Part 1:

Part 2:

Next, Stefan Groschupf, the co-founder and CTO of Datameer, discussed the challenges in social media analytics and how to overcome these using big data analytics built on Hadoop in his “Social Media: What’s Really the Buzz?” talk. The demo was very helpful in visualizing the true thought leads and influencers in social media conversations. These leaders and influences are becoming increasingly important, so that companies can better understand who is having an impact on their customers' buying decisions. This talk gave a very good perspective of how the power of Hadoop can be used to crunch large amounts of data and then visually rendered.

Part 1:

Part 2:

Part 3:

Finally, Matei Zaharia from UC Berkeley talked about Mesos: A Flexible Cluster Resource Manager. The talk highlighted Mesos features and how organizations can consolidate multiple application workloads into a single cluster. The demo showed off the benefits of Mesos and highlighted its ability to run multiple isolated instances of Hadoop on the same cluster. The fault tollerance of Mesos was successfully demonstrated too. Subsequent to the mail session, Matei and team talked about Spark, MapReduce-like framework that adds support for iterative jobs. Spark functional programming model similar to MapReduce was demonstrated capable of caching data between iterations making it very efficient for interactive analysis of big datasets. Spark in addition was demonstrated on the same cluster running alongside Hadoop.

Part 1:

Part 2:

Part 3:

We at Yahoo! embrace Hadoop, and are looking for exciting technologies and experiences you want to share. Please contact me via the Hadoop Bay Area User Group Meetup page.

Susheel Kaushik
Director, Product Management
Cloud Computing at Yahoo!

Satyam (a.k.a Daniel Barreiro) is a long-time YUI contributor and one of the most prolific, generous experts in the YUI forums. He is also the author of a new book on YUI 2.8.0, YUI 2.8.0: Learning the Library. This article in the “Ask Satyam” series was suggested by JoeDev. While its focus (like the focus of the new book) is mostly on YUI 2, many of the practices described here are applicable to YUI 3 as well — and to frontend development in general, regardless of your library of choice.

Before posting a question in the YUI Library forums, there are plenty of things you can do by yourself and, if you have your tools handy, you may find your answer all by yourself in no time. Besides, clean code is robust code, much less likely to break when subjected to stress. Good practices not only avoid fatal errors (the kind that drive you to the IRC channel or forums in search of help), but they surface warnings about minor errors and help you stay away from the fatal edge.

In this article, I’m going to take a look at some of those best practices. Some of these are specific to developing with YUI, but the vast majority apply to frontend development regardless of your choice of Ajax library.

JSLint

A trip through JSLint is part of the build process for YUI. JSLint, itself fully written in JavaScript, is like a compiler but without code generation. It will, however, produce many of the useful error messages and warnings that a compiler would. A browser’s JavaScript interpreter forgives many errors and assumes defaults you might be unaware of; JSLint forgives little, and it points you to better choices in your program. JSLint is available in many formats; the YUI Builder tool uses it as a standalone command line application, but you can also integrate it into Eclipse or whatever IDE or more-or-less capable editor you use — there is even a Yahoo! Widget for JSLint.

If the zillion lines of code in the YUI library can go through JSLint with no errors and few warnings (it can’t avoid a few things), so should your code. I find JSLint very helpful when I’m tired. Bean counting in the late evening is a waste of time; you are likely to miss the most obvious errors. JSLint doesn’t care what time of the day it is and will point you straight to that mismatched curly bracket or misspelled variable that’s at the root of the problem.

JSLint is most helpful if you keep your code clean from the start. If you’ve never used JSLint before and try it out on an application that is already hundreds of lines long, it will flood you with so many warnings that it will feel that it hampers your job more than it helps. However, that only happens the first few times. Once you learn to stay away from bad coding practices, JSLint diagnostics will be few and straight to the point; your real error, the one that’s driving you nuts, will clearly stand out. In the meantime, JSLint will teach you to make your code safe and robust.

So, don’t wait for a tough error to show up. If you have never used JSLint, try it with what you believe to be good code, it might not be as good as you assumed. As Douglas Crockford (author of JSLint) says, “JSLint will hurt your feelings.” But that’s a small price to pay for better code.

Global and unused variables

One of the things JSLint will warn you about is the use of global variables. There should be no global variables except those you know about, such as the ones created automatically like window or document and those for the libraries you are using such as YAHOO (in YUI 2) or YUI (in YUI 3). You may also want to create a single global for your own namespace.

Global variables are dangerous, and there are usually more of them than you’d expect; moreover, “extras” such as mashups and banners might add more of them. Since the window HTML DOM namespace can be omitted from compound names, window.name, window.length and window.location become the global variables name, length and location. Have you never used such variable names in your code? I don’t mean in their HTML DOM sense but as everyday field names in a table — like the name of a person, the length of an object or the location of a book on the shelf. What if you use those variables without declaring them? You might assume that length is initially undefined but, in fact, if you don’t declare your own local copy of it (and initialize it), length will be a reference to window.length. And if you assign something to location, you might accidentally cause your user to navigate away from the current page. Here, I am just giving examples of possible collisions with the browser’s built-in variables, but if you start adding libraries from other sources, the chance for collisions increases. The JavaScript syntax highlighter used in YUI’s examples uses the global variable dp as its root and the number of global variables any Google script might add when you insert a map or other tool in your page is impolite, to say the least.

It’s not merely that you want to step lightly with respect to other people’s variables — by staying out of the global namespace, you reduce the risk that they will step on yours (or that you’ll step on your own). With asynchronous threads weaving themselves around each other, as is often the case in modern JavaScript applications, you can’t really be sure the order in which your various pieces of code will execute and what global variable will be left at what value by whom. The only sane approach is to avoid them whenever possible.

Global variables might also point to a typo. If you have a global variable you didn’t mean to have and you have an unused variable with a similar name, you might have misspelled one of them: that is, you may have declared it with one name and used it with another different spelling. It can also mean that you have declared it after you have used it, which means that at execution time you might find your variable not initialized as you would have expected.

Using this

Using this is often troublesome for beginners because it’s easy to lose track of where we are in the scope chain. Until we get some practice, keeping track of scope can be an issue. Also, developing a coding style and a structure for the application helps greatly; after all, it’s all about knowing where you place things. If you learn to organize your code consistently and store state in logical places (this points to one such place), information in your program will be easy find. Any debugger will show you what this points to at each step. It’s always a good thing to check first if this is referencing the object you expect. A lot of times when we have a bug, it’s be because this is pointing to window. There are several situations that can produce this result.

If we have a method with an inner function, when we call that inner function it won’t get the scope of the caller but rather that of window. This example shows that the inner function doesn’t share the scope of the method within which it is contained:

var someObject = {
    outerFunction: function () {
        console.log(‘outer’,this); // prints outer Object {}
        var innerFunction = function () {
            console.log(‘inner’,this);// prints inner Window
        };
        innerFunction();
    }
};
someObject.outerFunction();

There are a couple of ways to fix this. In this example, we ask JavaScript to correct the scope using call():

var someObject = {
    outerFunction: function () {
        console.log(‘outer’,this); // prints outer Object {}
        var innerFunction = function () {
            console.log(‘inner’,this); // prints inner Object {}
        };
        innerFunction.call(this);
    }
};
someObject.outerFunction();

In the next example, we take advantage of the ability for inner functions to share the variables of containing functions. We create a variable self which we initialize to the value of this in the outer function. We can then use self in the inner function to refer to the outer function:

var someObject = {
    outerFunction: function () {
        console.log(‘outer’,this); // prints outer Object {}
        var self = this;
        var innerFunction = function () {
            console.log(‘inner’,self); // prints inner Object {}
        };
        innerFunction();
    }
};
someObject.outerFunction();   

Finally, with events, the scope of the listener is that of the element to which the listener is attached:

var someObject = {
    outerFunction: function () {
        console.log(‘outer’,this); // prints outer Object {}
        YAHOO.util.Event.on(‘button’,'click’,function () {
            console.log(‘inner’,this); // prints inner <button id="button">
        });
    }
};
someObject.outerFunction();

Unless we adjust the scope of the listener when setting it up

var someObject = {
    outerFunction: function () {
        console.log(‘outer’,this); // prints outer Object {}
        YAHOO.util.Event.on(‘button’,'click’,function () {
            console.log(‘inner’,this); // prints inner Object {}
        },this,true);
    }
};
someObject.outerFunction();

This also applies to YUI components. If we listen to a click event on a DataTable cell, a TreeView node or a MenuItem, the scope of the listeners will be those of the YUI component instance that owns the event — unless it is explicitly set otherwise.

Sandboxes for applications

Another good way to keep your code clean is to start with a clean skeleton. The style of coding JavaScript applications has changed over time. Nowadays, most developers use two different styles: one for applications and one for library components. Most YUI 2 examples reflect the old style, where we used YAHOO.namespace to create a namespace for our own code.

Nowadays, for our applications, we mostly use a single sandbox declared within the scope of an anonymous function that executes onDOMReady:

YAHOO.util.Event.onDOMReady(function() {
    var Dom = YAHOO.util.Dom,
        Event = YAHOO.util.Event,
        Lang = YAHOO.lang;

    var yourVariable = initialValue,
        moreOfTheSame = otherInitialValue;

    var myFunction1 = function ( …) {
        // body of function;
    };
    var myFunction2 = function (… ) {
        // body of function;
    };
    …
});

This technique has several benefits.

1. We check for the readiness of the DOM right from the start, ensuring that all the pieces of HTML that we might want to manipulate are present and safe to use.
2. The function provided to onDOMReady is not wasted at all; it’s the container of the sandbox and, because it’s anonymous, it does not pollute the global namespace.
3. We immediately start defining our variables, including aliases or short names for our most often-used objects. This has several other advantages of its own:
   1. We save some typing
   2. The YUI Compressor can compress our short names, whereas it cannot compress a global name like YAHOO or its properties such as util or Menu no matter how deep they might be. If they are anchored in a global name such as YAHOO, the whole branch is untouchable. Thus our already short names Dom, Event and Lang might be further reduced to A, B and C when Compressor is run at build time.
   3. The interpreter does not need to resolve over and over again the long names. Each dot in a name such as YAHOO.util.Event.onDOMReady is time consumed in a symbol table look up.
   4. All variables will be available anywhere inside this anonymous function, even to functions defined within, unless a variable of the same name was defined in them. Basically, it is as if a sub-global environment has been defined within, and all variables there will be available anywhere just by name.
   5. For the variables in the sandbox we don’t need to use this, which gives us such headaches when using traditional OOP technique of making even the main function an object.

4. We define our functions. We can do this with the var statement, as I did above or the function statement; there is a subtle difference but it is mostly irrelevant. I use the var statement to highlight that they are just as accessible as the other variables: we can access them anywhere in the sandbox.

Of course this relies on the ability of JavaScript to allow us to define functions within functions and on the fact that the inner functions have access to all the variables defined in the outer function. Basically, each function you define creates a new local environment accessible to any further functions within.

Sandboxing is the standard way of doing things in YUI 3:

YUI().use(‘module1′, … , function (Y) {
    // This is the sandbox
});

Namespaces for libraries

While the sandboxing technique is great for final applications, it’s not good for libraries. What happens in the sandbox stays in the sandbox, completely invisible to the outside world. However, when you define a library utility or component of your own, you want to re-use it, so it needs to be globally accessible (which is not the same as being purely global). Anything you define under YAHOO.example — e.g., YAHOO.example.myUtility — is globally accessible. You can access it by its full name once it has been defined. myUtility, as a member of the global YAHOO, is not global but it is globally accessible.

When we build libraries, we usually use the sandbox for our code and namespacing for sharing, like this:

(function () {
    var MyClass = function () {
        // this would be the constructor
    };
    MyClass.prototype = {
        // properties and methods
    };
    YAHOO.namespace('MyLibrary');
    YAHOO.MyLibrary.MyClass = MyClass
})();

We create a sandbox by defining everything within an anonymous function, which we immediately execute (see all those parenthesis there?, they mean 'take the result of defining this function and execute it'). Here, we don't wait for the DOM to be ready; libraries seldom do, since the application that uses them is responsible to check that everything is in place. Within the sandbox, we have the same advantages as with the previous sandbox. We can define short names for anything we use often, even for the class we are defining: none of them will be visible outside. Then, to make it globally accessible, we create a namespace of our own and place our recently created class in it.

YUI Logger

Let's say we have our code nice and clean, with no JSLint errors or warnings, but we still have problems to debug. YUI can be helpful in telling us what's wrong. For production code, we will usually load the minified versions of the YUI components, but there are also two other versions. The -debug.js version is the one that can help us uncover problems. For example, we might be using Dom Collection's method setStyle to, lets say, change the color of a block of text. The change doesn't happen and we can't find what's wrong. The file dom-debug.js has this helpful line, which is deleted in the -min version:

YAHOO.log('element ' + el + ' is undefined', 'error', 'Dom');

This is executed when setStyle tries to locate the element to be styled and does not find it. The error message will probably show a misspelled element id or some such error that is so hard to pick after a long day of staring at the same code.

It's easy to get the logger up and running; you just need to include its files:

<link type="text/css" rel="stylesheet" href="http://yui.yahooapis.com/2.8.1/build/logger/assets/skins/sam/logger.css">
<script src="http://yui.yahooapis.com/2.8.1/build/yahoo-dom-event/yahoo-dom-event.js"></script>
<script src="http://yui.yahooapis.com/2.8.1/build/dragdrop/dragdrop-min.js"></script>
<script src="http://yui.yahooapis.com/2.8.1/build/logger/logger-min.js"></script>

The basic yahoo-dom-event aggregate is most likely to be there already while the Drag & Drop optional dependency is handy to move the logger window out of the way. Then, just add:

var myLogReader = new YAHOO.widget.LogReader();

Then, it's a matter of loading the -debug versions of the component files and all the messages they produce will be shown. The amount of information can be overwhelming so, it is wise not to load all -debug versions at once. The Logger can also filter the information it displays and you may uncheck the Info category of messages to concentrate on Warn and Fail. The filters do not affect what is logged, only what is shown; Logger logs all messages, regardless of whether the LogReader displays them or not. There is only one message queue per application and logging will start as soon as the Logger component file is loaded, it doesn't matter whether it is shown or not.

Another power-user strategy for YUI debugging with the Logger Control is to take advantage of Logger's ability to leverage the browser's console (in supported browsers):

YAHOO.widget.Logger.enableBrowserConsole();

With this line of code, you can pipe all Logger messages to the browser console, which is a natural part of your workflow in development in any case.

The only tedious thing about the Logger is remembering to change the component files from the minified, aggregates and combos to the -debug versions. In this, as always, the Dependency Configurator is a great help; click on the -debug button on the top left and it will produce the correct list of files.

And, now that I've mentioned it:

Check your dependencies

It's a good idea when you start with YUI to pick an example that closely resembles what you want to do and modify it. As you add more features, you take bits and pieces from other examples and incorporate them into your developing application. One frequent source of problems are the dependencies. Many people paste the dependency files of each and every example into their application, duplicating some of them over and over again. Missing, duplicate or out of order dependencies might produce unexpected errors. Not having the YAHOO Global Object loaded before anything else is most certainly going to be fatal. At any rate, loading something twice will certainly increase the time it takes your page to load.

Finally, if you are using your own servers to load the YUI library, you might have the base path for your copy of the library wrong. If you have Firebug, go to the Net tab and check that there are no lines in red. Those would mean that a requested resource could not be found.

Debugging

The Firebug add-on for Firefox is still the best debugger around, plus it is free. By default, when the debugger is activated the "Break on all errors" option is on, which means that Firebug will stop at the point where it finds an error and show the error message and source code. Some of those errors will be the same that JSLint would have diagnosed, so JSLint is still the first place to start — especially because Firebug can only signal one error on each run while JSLint can signal them all at once. Some errors will only show up at runtime, so the debugger is the only option. Whenever you get to such a break, the first thing to check is this, a four-letter word of the trickiest sort.

When using a sandbox, debuggers will usually show local variables and arguments, but they will not normally offer to show the variables in containing functions, such as those declared in the sandbox and used from within the inner functions. The debugger will be able to show their value if you explicitly ask for them by name, but it will not show them automatically. Another alternative is to make them globally accessible. For example, since YAHOO.example is always available (when YUI 2 is on the page), if you want to keep an eye on a certain component at all times you can simply copy it there:

var myTree = new YAHOO.widget.TreeView('tree-container');
YAHOO.example.myTree = myTree;

We add the last assignment while debugging so we can keep an eye on the TreeView instance while debugging since myTree would normally be within the sandbox and invisible elsewhere.

Don't use members starting with underscore

Many people use the debugger to look for the names of variables or properties holding the information they need. Sometimes that information is stored in properties starting with an underscore, like _nodes. By convention, those are private properties, not for general use. JavaScript enforces no true concept of private or protected members; thus the leading underscore is the conventional way to signal the intent of the class developer to keep that property private. These properties can present a temptation (after all, you can see them there in the debugger), but there are a couple of good reasons why it is not wise to use them in your programs.

First, only public interfaces are supported. Since you are not supposed to use a property starting with an underscore, the class developer is free to change it at any time. The developer should have provided some public mechanism to access this same value; for example, a _nodes private property might have a getNodes() method or a nodes configuration attribute. Either of these would be the public interface to that same value. The second reason not to use properties with leading underscores is that their accessors might need to produce a secondary effect which accessing the private property would completely bypass. The component would then be left in an inconsistent, fragile state.

So, if you are using the debugger to look for a value and you find it in a property starting with a leading underscore, don't use it. Go instead to the API docs and look in the Methods index for some getXxxx() method with a similar name or in the Configuration Attributes index. Avoid using private variables.

Stack Traces

The YUI library is robust and reliable. If the debugger breaks within the YUI library, it's more likely that it is because of an error induced by your code that by some failure in YUI itself. JavaScript was designed to carry on, always coping with errors as best as possible. The YUI library does likewise. The -debugversions will have extra checks and will emit diagnostic messages, but they are all stripped out in the minified versions. If you see an error in a YUI file it is likely that it got there out of pure stubbornness, but don't try to find the error there. When the debugger stops it is not because it has found the cause of the error but because the JavaScript interpreter can no longer carry on.

Breaks in minified files are not helpful, if you get a message such as...

F is not an object in line 7 of dom-min.js

...it is pointless to ask in the forum about it. First of all, F is an alias generated by YUI Compressor to replace a longer, descriptive variable name in the original uncompressed file; secondly, the first few lines of a component file are usually taken by the copyright notice, which the YUI Compressor always respects. And since the YUI Compressor also deletes all new-line characters, as it does with other white space, all the executable code in dom-min.js is in lines 7, 8 and 9 — so the line number doesn't say much either.

That is when the Stack Trace tab in the right sub-panel of the Script tab of Firebug (or wherever you can find the stack trace in your debugger) comes handy. It tells you how you got there. Here is a screen capture from Firebug:

The item on top of the right panel, createEvent(), is the name of the function where the current statement is; the one below is the place where createEvent() was called; the one below that the function that called the previous one; and so on. Most debuggers will let you select any of those trace points and see how you got to where you are. Firebug also lets you check the value of the variables at each of the trace points. We can see in the left subpanel the yellow arrow pointing to createEvent(). We got lucky this time, as it will always point to the line containing the offending code, but that code might be anywhere within that line; fortunately, createEvent()happens to be at the beginning of the line. See the variables and arguments? B, G, L...there is no guessing what the original names could have been, as this is the YUI Compressor at work. However, note that YAHOO is untouched and the copyright notices are preserved.

You might not recognize many of the names in the stack trace, as they might be functions within functions within the YUI library itself. Eventually, you might see function names you recognize. In this image, I know showAlbums, which calls the constructor for the REDT class (which is code I wrote). I don't have a clue about anything else. That's where I will go looking...to the ones I know.

If you are not sure, you can go through the full list. If you see compressed code (assuming you have not compressed your own yet), simply ignore it, or switch to the non-minified version of the YUI source files. But focus primarily on those names you recognize that belong to your own code. Note that Firebug uses (?)() for traces it cannot name. Usually, your code in the sandbox will be signaled that way since the sandbox is contained in an anonymous function. At any such point in your trace, check the values of the variables you provided as arguments; some of them might not be what you assumed. Check them against the API docs to see if it is a valid argument value.

If the argument and variable values you see are inconclusive, at least you can place a breakpoint right before your code calls the YUI component. You might then be able to see how you got into the mess.

Stack traces are often ignored by developers but with JavaScript's tendency to keep digging itself into an ever deeper hole as it tries to carry on, it is good to be able to come up to the surface to look around. As with all debugging techniques, it might not always work, but it is nice to know it is there. Let's be honest, if you have read this far, you know what desperation is.

Asynchronous calls

A lot of the interactivity that characterizes Web2.0 apps is based on the ability to handle asynchronous events. In contrast with the original style of web applications, where every interaction involved waiting for a new page to be delivered from the server, the modern interactive applications involve setting things up and listening for any of several possible events to happen...and then responding to those events. We set listeners for clicks or keystrokes and or other programmatic events; these are usually intuitive. Asynchronous calls and their callbacks, however, often are harder to understand.

The most common error I see in implementations on the forums is to place code right after an asynchronous call such as Connection Manager's asyncRequest()or DataSource's sendRequest() methods, assuming that such code will be executed when the operation is completed. When you call such a function, you are just priming the operation, not executing it in full. No data will be available after the call to the async method. Only the request for such data would have been produced so far. The server must receive it and process it, and the reply (if it ever comes) is still off in the future. That is why for such asynchronous operations we use callback functions.

A callback is like the function we assign as a listener for an event such as a click on a button in a form; when the user clicks, the listener gets called. The callback on a asynchronous event is very much like this; just as we send a form for a user to fill in, we send a request message to the server, and just as we wait for the filled-in form to be submitted back to our program, we wait for the reply sent by the server to come back to us. We cannot know when the user will submit the form or the server send the reply; that's why we set what for a user action we call an event listener and for an asynchronous event we call a callback function. It might seem the server reply is fast if not instantaneous, but it means ages in CPU time.

Debugging asynchronous calls

Sometimes there is no alternative but to single-step through the code. Be careful when you get to asynchronous calls such as Connection Manager's asyncRequest() method, DataSource's sendRequest() or DOM's window.setTimeout(). These start an asynchronous request for data against the server or delay some action a given time and their callbacks get called when the data has been received or the time has elapsed. There are two issues to consider; first, the debugger won't step into the callback automatically. If you want to catch up with the reply, you have to put a breakpoint inside the callback function. Lots of people reach the point of calling the asynchronous method and expect the single-stepping to resume within the callback when the async operation is finished. This will not happen; the thread of execution does not flow automatically into the callback, and the debugger cannot be expected to figure that one out.

Second, when you reach the line with the asynchronous call, put the breakpoint in the callback and then let the application run. Usually, whatever goes after the async call is not really important; in fact, the async call is normally at the end of a function, since there isn't much to do until the reply arrives. Don't single-step over the async call, because the debugger will keep the JavaScript interpreter on hold and, when the reply arrives or the time lapses, it will be missed since the interpreter was frozen and unable to handle it. So, if you were clicking on Step Over, when you get to the async call, make sure you have the breakpoint in the callback and then click Run so the JavaScript interpreter is active and able to handle the reply.

For repetitive events such as callbacks to window.setInterval() or other time-critical operations, it is better not to put breakpoints in them. While you are on hold, many events will be missed. It is usually better to use console.log or the YUI Logger to simply signal that the event is happening and show a few critical values. Don't use window.alert() for the same reason; it puts a hold on the browser and the events you care about will be missed.

Conclusion

There are many tools to help us find out what is happening in our programs. With the proper tools at hand, we can find an error in less time than what it takes us to write a question in the forums. The first step, however, is to write good and reliable code and make JSLint an integral part of your development process.

The Stack Overflow Blog: “The Unix world loves to take sides. I don’t have to blog about this; Freud already did, in 1930. He called it ‘the narcissism of minor differences’”

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

This post was written and contributed by Frederick Townes, CTO of Mashable and Developer for the W3Total Cache plug-in, a Rackspace Cloud Tools partner.

WordPress is among the most popular open source publishing platforms on the planet. Out-of-the-box it performs in any hosting environment, but with W3 Total Cache, site performance is maximized leveraging the suite of tools it provides. User experience is one of the most important factors in site success and to improve it, reducing response times of your site is vital. Studies show that the faster your site loads, the longer visitors will stay, the more pages they will visit, the lower any learning curves become and the more favorably Google will rank your site. There are lots of optimizations W3 Total Cache does unobtrusively, without modifying plugins, theme, database etc.

Let’s take a look at some settings to get you started:

Page Cache:
Reduce load by caching pages and feeds after the first request (or even prime the cache if you prefer) and returning the saved file from either disk or memory.
Recommended Setting: Disk (enhanced)

Minify:
Remove white space, line breaks and other unnecessary characters to reduce file size so files are delivered faster. In the case of HTML, a minified file “feels” like it renders faster in the browser. While in the case of Cascading Style Sheets (CSS) and JavaScript (JS) files, they are also respectively combined to reduce the number of transactions required to deliver a given page. Use the help tool on the minify settings tab to easily find and prune the CSS and JS files to minify for a given theme. Avoid minifying statistics, advertising or similar JS files when making selections.
Recommended Setting:Disk

Database Cache:
Reduce read operations on the database by caching responses for a short time. Consequently, the time required to generate new pages is also reduced. The result is that the response time after users post new comments, perform searches or request a page that has not been cached is a low as possible.
Recommended Setting:Off, but varies by site, try disk and verify if performance increases.

Object Cache:
A “higher level” cache than the database cache, the object cache is a “runtime” cache containing the various pieces of data needed
Recommended Setting: Off, but varies by site, try disk and verify if response times decrease.

Content Delivery Network:
Latency is among the easiest ways to improve the user experience (reduce load time) of a site. Rackspace Cloud Files can easily be enabled using W3 Total Cache on your site to allow WordPress core files, theme files and even the media library to be hosted on the Cloud Files Content Delivery Network by Limelight Networks. Site visitors will then have a highly responsive experience when browsing. Once Cloud Files account settings have been populated and desired files exported (uploaded), everything then works transparently; when new attachments are added to the media library they are automatically hosted with Cloud Files.
Recommended Setting: Rackspace Cloud Files

Browser Cache:
The web browser can keep copies of files downloaded from the web sites you visit. In terms of user experience, this is the lowest latency cache possible. Once a file is downloaded specific headers can allow the browser to quickly check if the local version is up-to-date. If it is, the browser can then either render the page instantly or request fewer objects for a given page making the page load much faster.
Recommended Setting: Defaults are a good start, experiment with settings to achieve desired policies.

User Agent Groups:
If your site has mobile users or if it uses various themes for respective groups of users. The User Agent Groups tab of the plugin allows you to assign groups of browsers or devices to use a specific theme or even redirect them to another web site. This feature also makes sure that specific browsers/devices have respective caches and therefore the lowest response times possible.
Recommended Setting: Defaults are a good start, settings may vary per site.

Tips:

• Use the import / export settings option of the general settings tab of W3 Total Cache to backup, save and share settings files.
• Import external media library objects, then export to CDN to make sure that every possible file is hosted with the content delivery network.
• Review the usage section of the frequently asked questions tab of the plugin for tips on measure performance.
• Reduce the images used in your CSS file by taking advantage of the spriteme.org bookmarklet which makes creating image sprites very simple.
• Review the Best Practices for Speeding Up Your Site section of the Yahoo! Developer Network site for even more tips.

Have ideas on how to make W3 Total Cache better? Need tips optimizing your site’s user experience? Use the various support form options on the support tab of the pluginto get in touch!

How long it may take MySQL with Innodb tables to shut down ? It can be quite a while.
In default configuration innodb_fast_shutdown=ON the main job Innodb has to do to complete shutdown is flushing dirty buffers. The number of dirty buffers in the buffer pool varies depending on innodb_max_dirty_pages_pct as well as workload and innodb_log_buffer_size and can be anywhere from 10 to 90% in the real life workloads. Innodb_buffer_pool_pages_dirty status will show you the actual data. Now the flush speed also depends on number of factors. First it is your storage configuration – you may be looking at less than 200 writes/sec for single entry level hard drive to tens of thousands of writes/sec for high end SSD card. Flushing can be done using multiple threads (in XtraDB and Innodb Plugin at least) so it scales well with multiple hard drives. The second important variable is your workload, especially how dirty pages would line up on the hard drive. If there are a lot of sequential pages which are dirty Innodb will be able to use larger size IOs – up to 1MB flushing dirty pages which can be a lot faster than flushing data page by page.

So if we have system with single hard drive doing 200 IO/ssc, 48G buffer pool which is 90% dirty and completely random page writes we’ll look at 13500 seconds or about 5min per 1GB of Buffer pool size.
This is worse case scenario though it is quite common in practice to see shutdown time of about 1min per GB of buffer pool per hard drive.

Baron has written a nice post how to decrease innodb shutdown time which you may want to read on this topic.

---

Entry posted by peter | 8 comments

Add to: | | | |

The feature described in this article is available as of YUI 3.2.0pr2, and it will be a part of the upcoming 3.2.0 release. You can start playing with it today by following the code in this article.

SimpleYUI is a new way of loading and instantiating YUI 3. The SimpleYUI file contains a rollup of basic Ajax library functionality: DOM tasks, event abstraction, UI effects, and Ajax. Unlike other ways of loading YUI, SimpleYUI creates a YUI instance immediately upon loading and binds all included components to a global Y variable. Using SimpleYUI is easy:

<script type="text/javascript"
 src="http://yui.yahooapis.com/3.2.0pr2/build/simpleyui/simpleyui-min.js"></script>

<script>
 //Y is ready to use; no instantiation necessary:
 Y.one("#foo").addClass("highlight");
</script>

This isn’t a “lite” or de-contented version of YUI — you still have access to all of the power and features of the entire library when you start with the SimpleYUI file. However, SimpleYUI does provide a nice convenience by rolling up some common functionality and creating a global instance (Y) that’s ready to use immediately.

Work with the DOM

SimpleYUI gives you all of the standard DOM interactions in the YUI 3 API:

//get an element reference, add a click handler
Y.one('#demo').on('click', function(e) {/*handle click*/});

//add content to an element
Y.one('#demo').append('Additional content added to #demo.');

//listen for any click on any <li> that descends from #demo:
Y.one('#demo').delegate('click', function(e) {/*handle click*/}, 'li');

//move #demo to the location of any click on the document
Y.one('document').on('click', function(e) {
    	Y.one('#demo').setXY([e.pageX, e.pageY]);
	}
);

Create UI Effects

All of the UI effects that are part of the (new-for-3.2.0) YUI Transition module are available in SimpleYUI:

//fade #demo, then remove it from the DOM:
Y.one('#demo').transition({
    easing: 'ease-out',
    duration: 2, // seconds
    opacity: 0
}, function() {
    this.remove();
});

Ajax

SimpleYUI provides the IO module’s basic Ajax functionality:

// Make an HTTP request to 'get.php':
Y.io('get.php', {
    on: {
        complete: function (id, response) {
            var data = response.responseText; // Response data.
            // ... handle the response ...
        }
    }
});

The rest of YUI is just a use() away

You aren’t limited to what comes bundled with SimpleYUI. You can bring in any other YUI 3 component, YUI 3 Gallery module, or YUI 2 component with a simple use() statement at any time.

//Use drag and drop, which is not included in the SimpleYUI rollup:
Y.use('dd-drag', function(Y) {
    var dd = new Y.DD.Drag({
        node: '#foo'
    });
});

YUI 3 is good about loading anything you need whenever you need it; just master the use() statement and you’re always just one line of code away from anything in the library that you need.

Use SimpleYUI when…

• …you want to get started fast and learn the ropes of YUI;
• …you want to have basic Ajax library functionality available at any time in the life of the page without creating a new YUI instance.

Don’t use SimpleYUI when…

• …performance matters more than convenience;
• …you want to sandbox portions of your implementation into separate instances;
• …you want to be deliberate about when different components load and what the precise module/submodule makeup is on the page.

Squeeze has been frozen for some time now, and hopefully will be released by the end of the year, and today the Debian team has revealed the name of the next Debian release 7.0: Wheezy.

Just like all the previous releases, this is another character from Toy Story – wheezy – a rubber squeeze toy penguin with a red bow tie (that appears only in the 2nd movie). This will be the first character selected as a Debian version name which has not appeared in all the movies.

Source: http://lists.debian.org/debian-devel-announce/2010/09/msg00000.html

NoSQL fun… 5 minutes Q&A session covering NoSQL and relational databases; very funny. Check it out:

Testing controllers has traditionally been a hassle due to the requirements of setting up the bootstrap, the front controller and initiating the dispatch cycle. In June, Matthew addressed this with the release of Zend_Test_PHPUnit_ControllerTestCase way back in 2008.

Later, Matthew helpfully wrote an article on how to use it and I have used that as a starting point for the information here. (Thanks Matthew!)

The project I'm using is TodoIt, which is a simple ZF demo application, which needs unit tests.

Setting up PHPUnit

All your unit tests will live in the /tests folder. The ZF cli tool will create a phpunit.xml file for you, but you'll discover that it's empty! This is what it should look like:

<phpunit colors="true" bootstrap="./TestHelper.php">
    <testsuite name="TodoIt Test Suite">
        <directory>./</directory>
    </testsuite>

    <filter>
        <whitelist>
            <directory suffix=".php">../library/</directory>
            <directory suffix=".php">../application/</directory>
            <exclude>
                <directory suffix=".phtml">../application/</directory>
            </exclude>
        </whitelist>
    </filter>

    <logging>
        <log highlowerbound="80" lowupperbound="50" highlight="true" yui="true" charset="UTF-8" target="./log/report" type="coverage-html"></log>
    </logging>
 
</phpunit>

This file is used to configure phpunit itself and saves having to use command line options. As it's XML, it's fairly easy to read. The testsuites element is used to specify the testsuite we're going to test. In principle you can have many test suites; in this case, one is enough! The filter section is used to specify which files to use for code coverage reporting and the logging section is used to configure the reports.

We also specify TestHelper.php as the bootstrap. This mean that it is called for us and contains the necessary PHP setup we need to do in order to load and use Zend Framework. In effect TestHelper.php acts like public/index.php does for your web application. TestHelper.php looks like this:

<?php
// Based on http://weierophinney.net/matthew/archives/190-Setting-up-your-Zend_Test-test-suites.html

// PHP settings
error_reporting(E_ALL | E_STRICT);
date_default_timezone_set('Europe/London');

define('APPLICATION_ENV', 'unittesting');
define('APPLICATION_PATH', realpath(dirname(__FILE__) . '/../application'));

// Directories for include path
$root = realpath(dirname(__FILE__) . '/../');
$library = $root . '/library';
$models = $root . '/application/models';

$path = array(
    $library,
    $models,
    get_include_path()
);
set_include_path(implode(PATH_SEPARATOR, $path));

require_once 'Zend/Loader/Autoloader.php';
Zend_Loader_Autoloader::getInstance();

// Unset global variables
unset($root, $library, $models, $path);

As with public/index.php, we set APPLICATION_ENV and APPLICATION_PATH, update the include_path and then set up the autoloader. Now we're all ready to write some tests!

A controller test class

I place my controller test classes in tests/application/controllers to make them easy to find. (Model tests go in tests/application/models!). The TodoIt application has a login form in AuthController::indexAction() which is accessed via the /auth URL. We'll start by testing this form is displayed.

The controller's test class is called AuthControllerTest and lives in tests/application/controllers/AuthControllerTest.php:

<?php

// Call AuthControllerTest::main() if this source file is executed directly.
if (!defined("PHPUnit_MAIN_METHOD")) {
    define("PHPUnit_MAIN_METHOD", "AuthControllerTest::main");
}

require_once 'PHPUnit/Framework/TestCase.php';

/**
 * @group Controllers
 */
class AuthControllerTest extends Zend_Test_PHPUnit_ControllerTestCase
{
    public static function main()
    {
        $suite  = new PHPUnit_Framework_TestSuite(get_class($this));
        $result = PHPUnit_TextUI_TestRunner::run($suite);
    }
    
    public function setUp()
    {
        $application = new Zend_Application(
            APPLICATION_ENV,
            APPLICATION_PATH . '/configs/application.ini'
        );

        $this->bootstrap = $application;
        return parent::setUp();
    }

    public function tearDown()
    {
        /* Tear Down Routine */
    }

    public function testLoginDisplaysAForm()
    {
        $this->dispatch('/auth/index');
        $this->assertQueryContentContains('h1', 'Login');
        $this->assertQuery('form#login'); // id of form
    }
}

There's three things going on in here, so let's look at each in turn.

Firstly we set up the file to allow PHPUnit to run this file on it's own using the command line:

phpunit testsapplicationcontrollersAuthControllerTest.php

This is done by setting the PHPUnit_MAIN_METHOD constant to the static method AuthControllerTest::main(). The phpunit cli tool will then run this method which will in turn run this file as a test suite.

The methods setUp() and tearDown() are called before and after every test method and are used to ensure that we have a clean slate for each one. As we extended from Zend_Test_PHPUnit_ControllerTestCase rather than from PHPUnit_Framework_TestCase, we are able to leverage functionality specifically designed to make testing controllers easier. We use this in setUp() to set the property bootstrap to an instance of Zend_Application, which is then used in the tests themselves.

Each test is a method that starts with the word test, like this one:

    public function testLoginDisplaysAForm()
    {
        $this->dispatch('/auth/index');
        $this->assertResponseCode(200);
        $this->assertQueryContentContains('h1', 'Login');
        $this->assertQuery('form#login'); // id of form
    }

We start by calling dispatch() to run the correct action and then we use the various assert methods to check that the result is what we expect. The assertResponseCode method checks that we didn't error as the errorController will set the code to 500 or 404. We can then use the assertQuery methods to check what has been rendered to the response object. These use DOM paths to select a specific element. The call to assertQueryContentContains allows us to check the text within the H1 element is what we expect and the assertQuery just checks that the element is on the page.

That's it.

This is just the tip of the iceberg and I strongly suggest that you have a read of the documentation to see for yourself how many different assertions you can use to check that your code is performing as expected.

Few days ago I was working on a case where we needed to modify a lot of data before pushing it to sphinx – MySQL did not have a function to do the thing so I thought I’ll write MySQL Stored Function and we’ll be good to go. It worked! But not so well really – building the index, which was taking 10 minutes, was now taking 16 minutes. Then we added another MySQL function for different set of attributes and indexing speed went from 16 minutes to 26 minutes. I knew using UDF would be faster, but I had no idea how much. Have you ever wondered?

So what were the modifications we needed? It was couple very simple things – (1) two varchar columns needed leading nonalpha characters trimmed, so “123 ^&* and some text” would become “and some text”, and (2) same two varchar columns needed some double characters changed to single one so “Picasso” becomes “Picaso”, “Wesselmann” becomes “Weselman” and so on. Why we needed that is another story which this blog post is not about. Note however that only very small portion of data really needed to be modified.

Here are the two MySQL functions I wrote to do the job – ltrim_junk_mysql() and remove_dups_mysql(). Although processing single row seemed to be instantaneous, we needed to process much more than that – and that wasn’t as fast. For example, here’s how long it took to process 100k rows:

mysql> select ltrim_junk_mysql(author), ltrim_junk_mysql(title) from paintings limit 100000;
100000 rows in set (2.97 sec)

mysql> select remove_dups_mysql(author), remove_dups_mysql(title) from paintings limit 100000;
100000 rows in set (2.04 sec)

If you looked carefully at the second function though, you may have noticed I did not necessarily have to write a function, I could have written it as an SQL statement:

mysql> select
REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE(
REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE(
REPLACE( REPLACE( LOWER(author), 'aa', 'a'), 'bb', 'b'), 'cc', 'c'),
'dd', 'd'), 'ff', 'f'), 'gg', 'g'), 'll', 'l'), 'mm', 'm'), 'nn', 'n'),
'oo', 'o'), 'pp', 'p'), 'rr', 'r'), 'ss', 's'), 'tt', 't'), 'vv', 'v'),
'zz', 'z'),
REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE(
REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE(
REPLACE( REPLACE( LOWER(title), 'aa', 'a'), 'bb', 'b'), 'cc', 'c'),
'dd', 'd'), 'ff', 'f'), 'gg', 'g'), 'll', 'l'), 'mm', 'm'), 'nn', 'n'),
'oo', 'o'), 'pp', 'p'), 'rr', 'r'), 'ss', 's'), 'tt', 't'), 'vv', 'v'),
'zz', 'z') FROM paintings LIMIT 100000;
100000 rows in set (0.33 sec)

Doesn’t look nice, but it already executes more than 6 times faster which is interesting as it shows how much overhead you have by using mysql stored routines interface. So anyway, I asked my colleague Sasha to help me out by rewriting these as UDF functions. Here’s ltrim_junk() function and remove_dups(). Well, guess what:

mysql> select ltrim_junk(author), ltrim_junk(title) from paintings limit 100000;
100000 rows in set (0.13 sec)

mysql> select remove_dups(author), remove_dups(title) from paintings limit 100000;
100000 rows in set (0.17 sec)

So for ltrim_junk() function we got almost 23x improvement and for remove_dups – 12 times if comparing to stored function or 2 times comparing to just using available functions. With that speed I could even scan the whole table of 7 million records:

mysql> select count(*) from paintings where title != ltrim_junk(title);
+----------+
| count(*) |
+----------+
|   101533 |
+----------+
1 row in set (6.82 sec)

mysql> select count(*) from paintings where author != ltrim_junk(author);
+----------+
| count(*) |
+----------+
|    28335 |
+----------+
1 row in set (6.63 sec)

mysql> select count(*) from paintings where author != remove_dups(author) OR title != remove_dups(title);
+----------+
| count(*) |
+----------+
|  2720414 |
+----------+
1 row in set (11.19 sec)

Whereas using stored function used to take minutes!

I don’t mean to say stored functions are bad and you should now rewrite all your functions as UDFs – if you need to process just a few records for a request and you are not burning racks of CPUs to constantly do the job, the speed difference is really negligible. However in case like this one where we have to process many records constantly and every second counts, UDF can really save your day. If you need one and don’t feel confident writing C, you know who to call!

---

Entry posted by Aurimas Mikalauskas | 3 comments

Add to: | | | |

The YUI team is pleased to announce the GA release of YUI 3.2.0, a significant update to YUI’s JavaScript and CSS components. This release features a host of performance improvements and major new support for mobile devices with touch UIs.

The Gesture module in YUI 3.2.0 brings touch support to a wide range of YUI-powered interactions, including Drag and Drop and Slider.

There are two ways to get started with the new release:

• Use from the CDN: YUI 3.2.0 is available on the Yahoo! CDN via the 3.2.0 version tag — so you can reference the new release content via filepaths like http://yui.yahooapis.com/3.2.0/build/yui/yui-min.js. Switch to the seed file for the new release and all subsequent use() statements will load YUI 3.2.0.
• Download the release: Download YUI 3.2.0 from YUILibrary.com, including source code and examples for all components — including those new to this release.
• Explore the examples: YUI examples are a good way to get a feeling for how the library works and the range of functionality it covers.

Noteworthy Changes in YUI 3.2.0

George Puckett has rolled up the high-level list of changes in YUI 3.2.0; you can also check in on our progress addressing issues in the bug database. Here are some of the new and updated components featured in the 3.2.0:

• SimpleYUI has been introduced as a convenience module — in addition to rolling up a variety of YUI’s most essential features, it creates a global instance (assigned to the global Y) that’s available for immediate use. This isn’t a watered-down YUI…just a convenience for those who want to have a common rollup with standard Ajax-kit functionality.
  
• Intrinsic support for touch events has been added (mynode.on("touchstart", function(e) {});). We’ve also added a Gestures module with two bundled gestures — flick and move — that work with both touch- and mouse-driven devices. Check out the Gestures documentation and examples for ideas about how to start using Gestures.
• YUI’s loader now supports capability-based loading. This allows us to segregate, for example, IE-specific code into separate submodules and allow the Loader to bundle that code only for browsers that require it. We’re leveraging this new feature to avoid shipping IE-specific code in the Dom module to non-IE browsers, a performance/k-weight boost that will benefit all users of modern browsers with no code change required.
• YUI 3’s animation portfolio now supports transitions via the Transition module, providing browser normalization for this powerful, hardware-accelerated (where available) technique for handling transitions; check out this example for sample code. The Animation module, in its most basic form, has a streamlined dependency tree for modern browsers, significantly reducing the k-weight for simple animation in better browsers.
• YUI 3.2.0 debuts a new beta version of YUI’s CSS Grids component, and you can begin exploring this new approach to Grids in the preview release. The examples are the best place to start.
• We worked with Michael Johnston of the Yahoo! Mobile Engineering team to bring a new (beta) ScrollView widget to YUI 3.2.0. ScrollView provides a scrolling pane implementation familiar to users of native Apple iOS applications, emulating the elasticity of the element’s constraints when scrolled to the beginning or ending limit. You’ll see in the 3.2.0 examples for ScrollView that this component is device-neutral, working well with a mouse as well as with touch events on your Android or iOS device.
• The Uploader component from YUI 2 is now part of the YUI 3 family as well, debuting as a beta in 3.2.0.
• Editor (beta) for YUI 3: Editor debuts in YUI 3 as a base utility layer providing a foundation on which you can create a rich text editing control. This version of Editor does not contain a GUI of any kind — widget chrome will debut in 3.3.0.
• The History module that debuted with YUI 3.0.0, which was a port of the YUI 2 version, has been deprecated (it remains available in YUI 3.2.0 as history-deprecated). A new beta History utility debuts in 3.2.0, based on Ryan Grove’s History Lite module from the YUI 3 Gallery. A preview-release example from the new component is a good starting reference.
• The JSONP and YQL Query modules from the YUI 3 Gallery have become canonical components, debuting as beta in this release.

More to Come

We’re excited about the coming months in YUI development:

• YUI Conf 2010 takes place Nov. 8-10 in Sunnyvale. As usual, there will be a Crockford keynote, talks by some of the brilliant contributors to the YUI 3 Gallery (including Nate Cavanaugh and Eduardo Lundgren of Liferay), and sessions from the YUI core team.
• New components and features: We’re hard at work on YUI 3 versions of tools like AutoComplete, DataTable, Button, Charts and Editor, as well as some all-new projects. Expect a preview of 3.3.0, with all new work completed thus far, when you’re here for the conference in November.
• Yeti, YUI for Node.js, and more: The team continues to push the envelope with projects aimed at the most promising frontiers in frontend engineering. Reid Burke’s Yeti was just launched, allowing command-line cross-browser testing for anyone using YUI Test. Dav Glass’s work to make YUI 3 work seamlessly with NodeJS has been documented on the blog and will be an official YUI offering soon. And there’s more to come.

Feedback

Let us know what you think of the new release in the forums or @yuilibrary on Twitter.

The AWS Management Console now supports the Amazon Virtual Private Cloud (VPC). You can now create and manage a VPC and all of the associated resources including subnets, DHCP Options Sets, Customer Gateways, VPN Gateways and the all-important VPN Connection from the comfort of your browser.

Put it all together and you can create a secure, seamless bridge between your existing IT infrastructure and the AWS cloud in a matter of minutes. You'll need to get some important network addressing information from your network administrator beforehand, and you'll will need their help to install a configuration file for your customer gateway.

Here are some key VPC terms that you should know before you should read the rest of this post (these were lifted from the Amazon VPC Getting Started Guide):

Let's take a tour through the new VPC support in the console. As usual, it starts out with a new tab in the console's menu bar:

The first step is to create a VPC by specifying its IP address range using CIDR notation. I'll create a "/16" to allow up to 65536 instances (the actual number will be slightly less because VPC reserves a few IP addresses in each subnet) in my VPC:

The next step is to create one or more subnets within the IP address range of the VPC. I'll create a pair, each one covering half of the overall IP address range of my VPC:

The console shows all of the subnets and the number of available IP addresses in each one:

You can choose to create a DHCP Option Set for additional control of domain names, IP addresses, NTP servers, and NetBIOS options. In many cases the default option set will suffice.

And the next step is to create a Customer Gateway to represent the VPN device on the existing network (be sure to use the BGP ASN and IP Address of your own network):

We're almost there! The next step is to create a VPN Gateway (to represent the VPN device on the AWS cloud) and to attach it to the VPC:

The VPC Console Dashboard displays the status of the key elements of the VPC:

With both ends of the connection ready, the next step is to make the connection between your existing network and the AWS cloud:

This step (as well as some of the others) can take a minute or two to complete.

Now it is time to download the configuration information for the customer gateway.

The configuration information is provided as a text file suitable for use with the specified type of customer gateway:

Once the configuration information has been loaded into the customer gateway, the VPN tunnel can be established and it will be possible to make connections from within the existing network to the newly launched EC2 instances.

I think that you'll agree that this new feature really simplifies the process of setting up a VPC, making it accessible to just about any AWS user. What do you think?

--Jeff;

I've been reading the Proposal for Exceptions in ZF2 and like it. One thing that caught my attention was that it suggests that you can catch an interface. I hadn't heard of that ability before, so I pulled out my trusty text editor to have a play.

Consider this code:

<?php
namespace My; 
 
interface ExceptionInterface {} 
 
class SplExceptionClass extends InvalidArgumentException implements ExceptionInterface {} 
class ExceptionClass extends Exception implements ExceptionInterface {} 

class A {
    static function throwAnSplException()
    {
        throw new SplExceptionClass('oops');
    }
    static function throwAMyException()
    {
        throw new ExceptionClass('oops again');
    }
}

Within our My namespace, we have two exception classes and an exception interface that both classes implement. We also define a class MyA as a vehicle for throwing the exceptions. This is the basis of how ZF2 exceptions will work without all the actual component implementations :)

Let's do some testing to look at what happens:

Test 1

try {
    A::throwAMyException();
}
catch (ExceptionClass $e) {
    echo "Caught \My\ExceptionClassn";
}

As expected, this works. We catch the MyExceptionClass exception that was thrown.

Test 2

try {
    A::throwAnSplException();
}
catch (ExceptionClass $e) {
    echo "Caught \My\ExceptionClassn";
}

As expected, we fail to catch MySplExceptionClass exception that was thrown as it is not related to the MyExceptionClass that we are trying to catch.

Test 3

try {
    A::throwAnSplException();
}
catch (ExceptionInterface $e) {
    echo "Caught \My\ExceptionInterfacen";
}

This time we are catching the MyExceptionInterface and it works! This surprised me and is very handy.

We now have the ability with ZF2 to be able to use different exception classes to represent different error types rather than using string comparison and, at the same time, we can have a single catch() for when we don't need that level of granularity.

Eclipse Helios (3.6) and Zend Studio 7.1 (and above) include a new version of Eclipse PDT, labeled 2.2. This new release uses a new repository for storing code reference information. With these simple tips you can significantly improve the performance of this repository which will speed up code completion, searching for definitions, type hierarchy views and more.

Cloudkick, a Rackspace Cloud Tools partner, has released their support for Windows. This comes in perfect timing with the release of Rackspace Cloud Servers for Windows. So what does this mean? If you’re running Windows on Rackspace Cloud Servers, you can now leverage the power of Cloudkick’s management and monitoring tools to streamline deployment and ongoing management.

Benefits of using Cloudkick with your Windows deployment include:

• Auto-scale monitoring and fault detection
• Maximize ROI
• Automatic data aggregation
• Access to Simple, powerful tools

Click here to learn more.

Dan Di Spaltro,CTO Cloudkick, will be leading a webinar on how you can manage Windows in the Cloud. In this webinar, Dan will cover:

• Installation.See how to be up and running in minutes
• Custom Plugins. Design your own to monitor any metric
• Trending. Learn how to spot anomalies and prevent downtown
• Alerting. Set-up custom thresholds to alert for your specific needs

Date of the Webinar:Tues, Sep 21st
Time:1pm CDT

Click here to sign up.

I can’t tell you how many of you have told me you’d like to run smaller applications at lower cost on EC2. These applications are typically low traffic/low throughput—web applications, web site hosting, various types of periodic cron jobs and the like.

I’m happy to say we have now built an instance type exactly for these purposes, called Micro instances, starting at $0.02 (two cents) per hour for Linux/Unix and $0.03 (three cents) per hour for Windows.

Micro Instances (t1.micro) provide a small amount of consistent CPU resources and allow you to burst CPU capacity when additional cycles are available. They are available now in all Regions. You can buy Reserved Micro Instances and you can acquire Micro Instances on the Spot Market. Interestingly enough, they are available in both 32 and 64 bit flavors, both with 613 MB of RAM. The Micro Instances have no local, ephemeral storage, so you'll need to Boot from EBS.

CloudWatch can be used to watch the level of CPU utilization to understand when the available CPU bursting has been used within a given time period. If your instance's CPU utilization is approaching 100% then you may want to scale (using Auto Scaling) to additional Micro instances or to a larger instance type. In fact, at this low a price you could run CloudWatch configured for Auto Scaling with two Micro instances behind an Elastic Load Balancer for just under the price of one CloudWatch-monitored Standard Small instance.

While designed to host web applications and web sites that don't receive all that much traffic (generally tens of requests per minute, depending on how much CPU time is needed to process the request), I'm pretty sure that you'll be able to put this new instance type to use in some interesting ways. Here are some of my thoughts:

1. DNS servers, load balancers, proxies, and similar services that handle a relatively low volume of requests.
2. Lightweight cron-driven tasks such as monitoring, health checks, or data updates.
3. Hands-on training and other classroom use.

Feel free to post your ideas (and your other thoughts) in the comments.

Update:The AWS Simple Monthly Calculator now includes the Micro instances. The calculation at right illustrates the costs for a three year Reserved Instance running Linux/Unix full time.

-- Jeff;

Area 51 is filling up with thousands of ideas for new Stack Exchange sites, and a pretty clear pattern has started to worry us: too many ridiculously niche proposals, overlapping proposals, and proposals that are already covered by an existing site.

Merging Season

 

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

“But customers don’t know what they want!”

It’s an anguished cry that I have heard often from startup founders. In a way, I don’t blame them. I’ve been there myself. If we’re not attempting something truly new and innovative – what’s the point? If we’re just going to conduct the world’s biggest focus group to decide what to do, why couldn’t any old idiot do it instead? Isn’t the whole point of devoting our life to this enterprise to show the world that we have a unique and visionary idea?

I remember one conversation with a visionary quite well. He had just come back to the office after a few days away, and he was filled with big news. “I have incredible data to share!” which was pretty unusual – a visionary with data? He carefully explained that he had conducted a number of one-on-one customer interviews, showing them an existing product and then documenting their reactions. His conclusions were well thought out, coherently based in the data he was presenting, and painted an alluring picture of a new way forward. His team almost exploded on the spot.

“That’s the same idea you’ve been pushing for months!” “What were the odds? Customers explained to you that we need to do exactly what you wanted to do anyway? Wow!” It was an ugly scene.

We all know that great companies are headed by great visionaries, right? And don’t some people just have a natural talent for seeing the world the way it might be, and convincing the people around them to believe in it as if it was real?

This talent is called the reality distortion field. It’s an essential attribute of great startup founders. The only problem is that it’s also an attribute of crazy people, sociopaths, and serial killers. The challenge, for  people who want to work with and for startups, is learning to tell the difference. Are you following a visionary to a brilliant new future? Or a crazy person off a cliff?

True visionaries spend considerable energy every day trying to maintain the reality distortion field. Try to see it from their point of view – none of the disruptive innovations in history were amenable to simple ROI calculations and standard linear thinking. In order to do something on that scale, you need to get people thinking, believing, and acting outside the box. Their greatest fear is categorically not that their vision is wrong. Their real fear is that the company will give up without ever really trying.

This is where data, focus groups, customer feedback, and collaborative decision-making get their bad rap. In many cases, these activities lead to bad outcomes: watered down vision, premature abandonment, and local maxima.

When visionaries say “but customers don’t know what they want!” they are right. That’s the problem with false dichotomies: each side has a kernel of truth within it. You cannot build a great product simply by obeying what customers say they want. First of all, how do you know which customers to listen to? And what do you do when they say contradictory things?

And yet, the people who resist visionaries also have a point. Isn’t a bit scary, maybe even suicidal, to risk everything on a guess – even if it is emotionally compelling?

Like all false dichotomies, if either side “wins” this argument, the whole enterprise loses. If we just follow the blind mantra of “release early, release often” and then become purely reactive, we’re as likely to be chasing our tail as to be making progress. Similarly, if we pursue our vision without regard to reality, we’re almost guaranteed to get some aspects of it wrong.

The solution is synthesis: to never compromise two essential principles. One, that we always have a vision that is clearly articulated, big enough to matter, and shared by the whole team. Second, that our goal is always to discover which aspects of this vision are grounded in reality, and to adapt those aspects that are not.

A vision is like a sculpture buried in a block of stone. When the excess is chipped away, it will become a work of art. But the challenge in the meantime is to discover which parts are essential, and which are extraneous. The only way to do this is to continuously test the vision against reality and see what happens.

So what should you do if you find yourself working with a visionary? Almost every successful visionary has found partners to work with that help them stay grounded in reality. To do this you have to find ways to be supportive of the vision at the same time as reporting the bad news about where the vision falls short. I recommend a mantra that I learned from Steve Blank: always consider your job to find out if there is a market for the product as currently specified. Don’t try and change the vision every time you get new data. Instead, get out of the building and look for customers for whom your product vision is a slam-dunk fit. If and only if, after exhaustive searching, you cannot find any customers that fit the profile, is it time to have a serious conversation about whether and how the vision should be modified (a pivot).

And what should a good visionary do to help find synthesis? Based on the successful visionaries I have had the opportunity to work with up close, I'd like to offer two suggestions for the role a visionary should take on:

1. Identify an acute pain point that others don’t see. It’s important to specify the vision as much as possible in terms of the problem we’re trying to solve, rather than a specific solution. (Or, to use Clay Christensen's formulation, of the "job" customers are hiring us to do.) Even though the visionary surely has some concrete ideas which are to be tried, he or she should always be asking, “would I rather solve the problem, or have this specific feature?”
   
2. Hold the team to high standards. Despite Steve Jobs' incredible talents, he doesn’t personally design and ship every Apple product. It’s much more likely that his main function is to hold everyone who works for him to the same high standard. Once they’ve agreed to try and solve a dramatic problem, it’s the visionary’s job to hold each provisional result up to the light of that vision, and help the team remember that although trade-offs and compromises are always necessary – the real payoff is in solving that acute pain. This can help avoid the trap of the false negative: even if the first few iterations don’t get it right, the vision inspires us to learn from our failures and keep trying.

Let me close with a specific story of a visionary at work. I’ve heard from several sources a story about Jeff Bezos and the invention of one-click shopping. It may be apocryphal, but it’s illustrative anyway. Amazon had tasked a team with building their new one-click shopping feature, which was designed to reduce the friction required to make an impulse purchase. The purpose of naming the feature “one-click” was to clearly communicate to everyone the vision of maximum simplicity. When Bezos was meeting with the team to review their first version of the feature, so the story goes, after he clicked to make his purchase, he was prompted with a confirmation dialog box. He had to click “yes” to continue. In other words, one-click shopping required two clicks!

Now, it’s really important to see this story from both sides. Bezos was surely infuriated that the team had missed so obvious a point about his vision. But see his team’s point of view: they were immersed in a culture of protecting the customer. It was probably considered too dangerous to let someone “shoot themselves in the foot” and make an unintended purchase that could have serious economic consequences.

But by actually building a version of this feature, and doing some simple testing with customers and with Bezos, this team surfaced an issue that probably wasn’t really clear in Bezos’ vision from the get-go. Namely, how are we going to handle the case of customers one-clicking by accident? The synthesis solution is so simple, I’m sure it seems obvious in retrospect (and I’m sure dozens of people, for all I know including Bezos himself, are now sure they came up with it on their own): since mistakes are the uncommon case, give the customer several opportunities to realize and correct them after the fact, rather than trying to prevent them with a confirmation dialog box.

Those are the attributes I admire in successful visionaries: a determination to see the vision through, holding their teams to high standards, and a commitment to iterate in order to get there.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

One of the sayings I hear from talented managers in product development is, “good enough never is.” It’s inspirational, always calling the team to try harder and do better. It works to undermine excuses for poor or shoddy work. And, most importantly, it helps team members develop the courage to stand up for these values in stressful situations. Especially in teams that are managing by objectives (or OKR's), the pressure to deliver is intense. Under such pressure, the temptation to cut corners, to quit prematurely, or to hand off shoddy work to another department is overwhelming. It requires courage to stand up and say: "this work is simply not good enough. Sure, we could get away with it, but that's not how we work." Good managers work hard to create an environment where this courage thrives.

We’ll be on the road in October evangelizing Rackspace hosting and cloud computing. If you will be at any of these events or reside in any of the cities below, we’d love to meet you.

Warmgun

The 1-day conference about measurable design.
October 8th 2010
San Francisco, CA

Robert Collazo, Professional Services Manager at The Rackspace Cloud, will be leading a session on Scaling your Websites.

__________________________________________________________________________________

SNW

Featuring more than 120 educational sessions and presentations by to IT management experts covering today’s hottest IT topics – from cloud computing to energy efficient datacenters to virtualization to storage – and so much more.
October 11-14th
Dallas, TX

__________________________________________________________________________________

BlogWorld Expo

Social Media Conference where attendees learn about content creation, distribution and monetization strategies.
October 15th-16th
Las Vegas, NV

__________________________________________________________________________________

Open SQL Camp Boston 2010

An un-conference style get together for people interested in open source databases, including non-relational databases, database alternatives like NoSQL stores, and database tools such as Gearman.
October 15th – 17th
Cambridge, MA

__________________________________________________________________________________

Garter IT Expo

This is an annual gathering of CIOs and senior IT leaders. It delivers independent and objective content with the authority and weight of the world’s leading IT research and advisory organization.
October 17th – 21st
Orlando, FL

__________________________________________________________________________________

Interop

Interop drives the adoption of technology, providing knowledge and insight to help IT and corporate decision-makers achieve business success.
October 18th -22nd
New York, NY

Andy Schroepfer, VP of Enterprise Strategy, will be leading the following sessions and will be a part of the Keynote on Thursday:

Online Dating Meets SaaS Business Apps
Get Clear About Business Apps in the Cloud

__________________________________________________________________________________

NY XPO for Business

This event is the largest and most profitable B2B networking conference in the Northeast, featuring 30+ educational seminars and over 300 exhibits packed with hundreds of business products and services.
October 20th
New York, NY

__________________________________________________________________________________

SpiceWorld IT Conference 2010

A full 2 full days of topics ranging from intermediate and advanced how to’s on using Spiceworks to manage everything from configuration changes and purchasing to remote sites and virtualized environments to everyday IT best practices such as SMB IT storage, security and virtualization essentials.
October 21st -22nd
Austin, TX

In case you missed the Rackspace Cloud Files’ CDN webinar a few weeks ago, the recording is below for your review.

In this webinar, Jason Thibeault, Sr. Director Cloud Strategy, Limelight Networks, and myself, Product Manager for Rackspace Cloud Files, take a look at both the storage market as a whole and Rackspace customers to show how CDNs can be used to bring data to companies with modern needs.  The webinar compares Cloud Files and Limelight speeds to others in the market, as well as highlights some of the major events that Limelight has backed, proving their stake as an industry leader in the CDN space.

This is a perfect webinar for you if you still find yourself asking, “Why would I need a CDN?”  Look out for more webinars from Rackspace Cloud Files in the coming months, and don’t be shy to tell us what you’d like to learn about.  We’re here to help!

The first Stack Exchange site to make it all the way through the community creation process is now live and out of beta!

webapps.stackexchange.com is a place to get help with web applications. Want to know how to email huge files? How to delete your Facebook account? How to secretly follow someone on Twitter? How to backup Wordpress blogs, ahem, Jeff? This is the site.

(Updated: Only days after launching, we changed the name from "nothingtoinstall.com" to "webapps.stackexchange.com" because nobody really liked that name!)

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.