Adam Leach

Thank you, 2010.

I'm excited for 2011, and I want to share some of my plans for the coming year. But before I do, I also feel obligated to take a look back on my plans for last year.

I can't believe a whole year has passed since I wrote a blog post called "Towards a new entrepreneurship" laying out my priorities for 2010. Back then, I wrote:

"[Here is] an idea that I don't think is too widespread yet: that entrepreneurship is an industry. Sure, when entrepreneurs create startups that grow up into mature companies, they become part of an established industry, with its own ecosystem, norms, partners and best practices. But until that happens, we entrepreneurs have our own ecosystem, of investors and service providers, norms and even some "best" practices. The two ecosystems have diverged significantly in the past fifty years - and especially in the past ten. The reason is that the underlying theory that powers established business, the theory of general management, is increasingly inadequate for managing startups. And yet, so far, we lack a coherent theory to replace it. My belief is that the lean startup is that theory. Together, we are part of a movement that is redefining entrepreneurship."

This idea has become a reality in many ways this year. Our ideas have entered the mainstream of startup thinking and even the popular culture. I mean, who ever thought we'd see this cartoon in the New Yorker magazine?


Lean Startup Meetups are now in more than 75 cities, with more than 12,000 combined members. More and more, I am meeting entrepreneurs and managers from companies large and small who agree on this one point: entrepreneurship is management. By applying the same scientific principles that gave rise to general management in the first place to entrepreneurship and innovation, we are unleashing incredible creativity. But we still have a long way to go.

In that spirit, I want to review the four priorities I laid out at the start of this year. For 2010, I announced four main projects. Here's how I laid them out (in their original embarrassing order), and here's how each one turned out.

1. The Lean Startup Cohort program. Verdict: FAILURE. This seemed like such a promising idea at the time. Take a small number of high-growth companies and have them pay a premium price to learn from me and from each other how to apply Lean Startup ideas in depth. My main hypothesis was that making the program expensive would act as a quality filter, and that if we could find smart, committed companies to participate, they would all benefit tremendously. Thus, I assumed the biggest risk was finding participants who could afford the price.
   
   Unfortunately, I was completely wrong. Finding participants was no problem; the program quickly filled up. And the quality of participants was way higher than I imagined. And yet, when we actually started to run the program, it still failed. Teaching Lean Startup concepts in a fixed order really didn't work, since all active companies face different challenges at different times. And even in a strict, high-quality filtered room, most companies didn't want to share their problems and internal data, nor did they particularly want to engage with other companies' problems. In retrospect, that should have been obvious to me - as an entrepreneur, I would never have had the patience for a program like that.
   
   What's that you say? Even "gurus" have to get out of the building, build a minimum viable product, and pivot? Why, yes, they do. Embarrassing, but at least we failed fast. (Peter Drucker thought people used the term guru because it was easier to spell than charlatan.)
   
   
2. Teaching in academia. Verdict: MIXED. I started the year co-teaching a Lean Startup class for MBA's at Berkeley with Steve Blank. In some ways, it was a big success: the class was oversubscribed, had a record number of auditors, and received positive reviews. But the experience left me with doubts about whether that is the right way to engage with academia, for me.
   
   I strongly believe academia has an important role to play in transforming the practice of entrepreneurship. Luckily, Steve has been leading the charge to bring a new way of teaching entrepreneurship into academic programs, and 2010 saw the debut of his Durant School of Entrepreneurship at sllconf, as well as new programs like the Lean Launch Pad at Stanfordand the Business Model Competition at BYU.
   
   I believe significant new research also needs to be done. What we know today is just the tip of the iceberg about this new entrepreneurial management. How many of our beliefs are just tactics that sound good, or that only work in certain situations? Much more is needed, and 2011 will see the first few buds of that research project flower. My colleagues at Harvard Business School will debut a new Lean Startup-themed course for MBAs this spring, as well as a new $50,000 Minimum Viable Product Fund. As part of that project, HBS has commissioned a series of new case studies on Lean Startup practices, both in and (importantly) outside the software industry. (You can see a little taste at Jeffrey Busgang's blog here.) I've also begun a collaboration with Nathan Furr at BYU to research actual practitioners, following them over time with an eye towards discovering ways to test some of our beliefs about Lean Startup ideas empirically. You can follow our work (and volunteer to be studied) here.
   
   Also along these lines, I've worked with a variety of collaborators to produce case studies right here on Startup Lessons Learned. Hopefully, more will come in the new year. You can see our efforts so far.
   
   
3. Startup Lessons Learned Conference (sllconf). Verdict: SUCCESS. This was a project I almost didn't do this year, because the prospect made me so nervous. Boy am I glad I did. I still receive regular feedback from people who were there live or in one of our 60+ simulcast locations around the world. It was always a dream of mine to produce a conference where knowledge - not hype - was king, where information was presented in a useful order, and where success theatre and vanity metrics were banned. I believe we succeeded on all three counts.
   
   In case you missed it, here's a little taste of the event itself, courtesy of my friends at Micro-Documentaries:
   
   
   
   And don't forget, you can watch full video of the entire conference courtesy of our sllconf Justin.tv channel.
   
   In 2011, we will do sllconf again, probably in mid-May. As always, I will look to you readers for guidance and suggestions of what we should do different. Stay tuned for details. If you are interested in speaking or mentoring at sllconf 2011, we will accept suggestions and nominations. If you would like to nominate someone, please post a video of them giving a talk (with slides if possible).  Grainy low-def youtube videos are perfectly adequate. We had far too many submissions last year on behalf people I didn't know. I had to be confident they would meet the standards I laid out above, but I couldn't take the time to meet them all. Therefore, if you'd like to speak this year at sllconf, a great way to get a leg up would be to speak at a Lean Startup Meetup, and ask someone to record the session. And if you are a meetup organizer, and have had a great speaker who you'd like to see at sllconf 2011, please let me know.
   
   In other conference news, we'll also have an event at SXSW. We'll make details available soon, I promise. If you're going to be in town for SXSW, and might like to join as a speaker, sponsor, or attendee, please let me know.
   
   
4. Writing a book. Verdict: TBD. I am in the final weeks of preparing a manuscript for The Lean Startup Book which will be published by Crown (one of the largest business book publishers in the world) in 2011. I sincerely hope you'll like the final result; it has been a labor of love for me all year.
   
   Deciding to publish this book through traditional channels took a lot of thought. I believe it is time for our movement to Cross the Chasm into mainstream awareness. Our early successes have been impressive, but we are still just at the beginning. In my talks all this year I have been exhorting audiences to Stop Wasting People's Time. Our modern economy is full to the brim of waste: building products that have few customers, that produce negative returns for investors, or companies stuck in the land of the living dead. And yet, the people who are responsible for this waste are not generally early adopters of new ideas about entrepreneurship. They are not scouring blogs for the latest gems in innovation thinking. They are overwhelmed, doing the best they can, and get information from only a few sources. They don't want avant garde advice, they want to be reading the same things everyone else is reading.
   
   My belief is that, in order to reach this mainstream audience, we need to produce a book that is accessible to them, and then make that book a bestseller. That's one of my main goals for 2011, and I will be asking you to help many, many times in the coming year. I hope you'll continue to support me as you have this past year.
   
   As a reader, the rational thing to do with a new book is to wait until the book comes out, see if your friends and colleagues read it, and if they do, see if they think it's any good. That's classic mainstream customer thinking. Hopefully, the early adopters and visionaries among you will disregard this advice, and agree to pre-order the book instead. The more of you who do that, the more people we'll be able to reach when it debuts next year. Remember, mainstream customers will be looking to you to see if it's worth buying.
   
   You can pre-order it from me directly, or get an even better price at Amazon.
   
   (If you'd like to help, I'm still looking for test readers, case studies, and - most importantly - help bringing traffic to the book website. We're running constant A/B tests there; anyone who is able to donate traffic, ads, or a link from your own blog/website will have my gratitude.)

So that was 2010. I believe 2011 will be even better.

It's an auspicious time. Entrepreneurship is in a new renaissance. There are more startups operating today than at any time in history. New ideas about entrepreneurship are in the air. And the dominant management paradigm of the past century has run its course. Literally.

2011 will mark the one hundredth anniversary of the idea of management. I date its origin to the publication, in 1911, of Frederick Winslow Taylor's The Principles of Scientific Management, one of the most important management books ever written. Management's second century will be very different than its first. Our problems are more complex, faster moving, and we face greater uncertainty. In other words, we need entrepreneurs to solve them. I'm excited to see what comes next.

I hope you've all had a happy holidays, and I wish you the best for a new and exhilarating New Year. Here's to 2011!

Two quick things:

Permalink | Leave a comment  »

Symantec released it's MessageLabs Intelligence 2010 Annual Security Report on December 7th, 2010. As I read through the information they provided I pondered what I have seenover the last year. I am the Security Administrator for a small ISP in the midwest. We have had a very busy year and it seems like we are always trying to cleanup a mess that one of our customers has made. From virus infected computers to buggy website design we have seen some pretty interesting events take place. Some of the botnet activity that the report discusses did indeed originate from our customers infected machines. I struggle everyday with exactly how to get a handle on the problem. Each and every time I get abuse reports indicating that one of our customers has a computer/mail server sending out spam, I use it as an opportunity to educate the customer. Many of the customers that I talked to were not using anti-virus or they had anti-virus but didn't realize that it has to berenewed every year. Many of the customers were using programs that are prone to opening up a big old hole on their computer inviting the bad guys to come on in. Looking at the information provided in the report we see that Symantec estimates the total number of botnets worldwide is between 3.5 and 5.4 million. According to their findings Rustock still dominates the botnet world and is responsible for some 44 billion spam emails a day. Grum and Cutwail are in second and third place and are believed to be responsible for a large amount of the malware infections that have taken place over the last year.
www.symantec.com/about/news/release/article.jsp
The report provides some statistical observations for 2010.
Top Trends in 2010
Web Security: For 2010, the average number of new malicious websites blocked each day rose to 3,066 compared to 2,465 for 2009, an increase of 24.3 percent. MessageLabs Intelligence identified malicious web threats on 42,926 distinct domains, the majority of which were compromised legitimate domains.
Spam: In 2010 the annual average global spam rate was 89.1 percent, an increase of 1.4 percent on the 2009. In August, the global spam rate peaked at 92.2 percent when the proportion of spam sent from botnets rose to 95 percent as a new variant of the Rustock botnet was seeded and quickly put to use.
Viruses: In 2010, the average rate for malware contained in email traffic was 1 in 284.2 emails (0.352 percent) almost unchanged when compared with 1 in 286.4 (0.349%) for 2009. In 2010, over 115.6 million emails were blocked by Skeptic representing an increase of 58.1 percent compared with 2009. There were 339.673 different malware strains identified in the malicious emails blocked. This represents more than a hundred fold increase over 2009 and is due to growth in polymorphic malware variants.
Phishing: In 2010, the average ratio of email traffic blocked as phishing attacks was 1 in 444.5 (0.23 percent), compared with 1 in 325.2 (0.31 percent) in 2009. Approximately 95.1 billion phishing emails were projected to be in circulation in 2010.
The report says It is predicted that in 2011 botnet controllers will resort to employing steganography techniques to control their computers. This means hiding their commands in plain view perhaps within images or music files distributed through file sharing or social networking web sites. This approach will allow criminals to surreptitiously issue instructions to their botnets without relying on an ISP to host their infrastructure thus minimizing the chances of discovery.
What are you planning on doing in 2011 to minimize the impact on your network and to prevent your computers from being the victim? What do you anticipate your biggest threat to be for 2011?
Deb Hale Long Lines, LLC (c) SANS Internet Storm Center. http://isc.sans.org Creative Commons Attribution-Noncommercial 3.0 United States License.

Hi everyone, Sorry for taking so long to get this new version of Agnitio released! A combination of snow wrecked travel plans, Christmas and the cold I’ve currently got meant the work I’d planned to do on Agnitio got delayed, better late than never though! I was very happy to see that v1.0.0 had over [...]

Hy fellows,

first I want to wish you all a good start into this new decade.
Then I wanted to inform you about some news for Zend Framework.

Within the last days I began a complete rework of the I18n core for ZF.
The first class which will be reworked is Zend_Locale as it’s the base class for I18n within ZF.

The following changes will be done for Zend Locale 2.0:

CLDR update to 1.9
This integrates the most actual locale database within ZF

Usage of a fallback locale
This locale can be set and used as fallback when the wished locale is not supported

Usage of a fixed locale
This locale can be set and will be used regardless of any other locale which the accessing user wants
This removes the workaround from Zend_Application

Add locale providers as base
This allows to use other locale providers than CLDR
This will remove Zend_Locale_Format and integrate it into the used provider

Rework CLDR to be a provider

Add INTL as new provider
This allows to use INTL when available alone or in combination with CLDR because the INTL extension does not provide all informations like CLDR

Add a way to upgrade a locale
This would allow to provide informations for language locales when no region was given (f.e. when a user gives ‘en’ and wants to have informations from a region (f.e. currency))

Add script support
This allows to use locales which provide several scripts within the same language like Azerbaijani, English or Hausa.

That’s the actual plan for Zend_Locale 2.0 and will be the base for all other reworks.
Please note that all spoken will be done within Zend Framework 2.0 and not be migrated to 1.x.

Greetings
Thomas Weidner
I18N Team Leader, Zend Framework

Zend Framework Advisory Board Member
Zend Certified Engineer for Zend Framework

It’s been a while since I’ve written a technical post, so I thought, maybe I should write about what I’ve been working on for the past couple months…

I’ve been building a test and development infrastructure for The Wikimedia Foundation using OpenStack, and a number of other technologies. I’m not done yet, so I won’t get into any gory technical details (I promise I will later!). I will, however, give an overview of the architecture I’m aiming for.

Basic overview

We want a test and development infrastructure for a number of reasons:

1. We’d like to replace the Tesla infrastructure I built for the Usability Initiative project
2. We’d like to have an infrastructure where we can let volunteer developers and staff work collaboratively, and easily build infrastructures without relying on the limited resources of the (overworked) operations team
3. We’d like to be able to run realistic tests of our operational infrastructure so that we can be better prepared for outages, and so that we have an environment to safely train new operations staff
4. We’d like to have an infrastructure we can use to vet operations volunteers before we allow them access to the operational infrastructure
5. We’d like to have an infrastructure where developers can easily have root

With the above goals in mind, the infrastructure needs to handle most things automatically. We (operations) don’t want to have to manage user accounts. We don’t want to have to create virtual machines for people. We don’t want to have to manage DNS entries, or IP addresses. We do want an infrastructure that is close to our production environment, but is flexible enough to let developers add infrastructure without our help. We do want an infrastructure that lets developers prepare their projects for running inside of the production cluster by using the exact same processes as the operations team.

I’m creating an infrastructure to handle this, and here’s the basic architecture:

1. OpenStack as the virtualization technology
   1. Four nodes: 1 controller node, and 3 compute nodes (should be able to run roughly 100-120 VMs)
   2. Handles VM creation and scheduling on compute nodes
   3. Handles IP address allocation
   4. Has EC2 and Openstack APIs
   5. Gets user account information from LDAP
   6. Stores IP/VM information in MySQL
2. PowerDNS with an LDAP backend for DNS entries
   1. Currently using “strict” mode for this
   2. Each VM gets a DNS entry based on the name of the VM, and a cname record based on the “instance id” provided when the VM is created
   3. Can handle multiple DNS domains
3. Puppet with an LDAP backend for puppet nodes
   1. Node entries stored in LDAP so that users can easily select server types when creating VMs
   2. Puppet manifests, files, and templates stored in SVN or git repository
      1. Everyone with an account will be able to modify puppet, but changes will need to be merged into the main branch by an operations team member
      2. Ideally branches can be merged into the production cluster’s puppet repository as well
4. MediaWiki as the virtual machine manager
   1. Manages VM creation/deletion/modification, DNS, Puppet, user accounts, sudo access, user groups, user SSH keys, and OpenStack projects
   2. Using the OpenStackManager extension and the LdapAuthentication extension
   3. Progress on this is going well. Basically I just need to add localization and more error checking for this to be at a usable level (if you’d like to help with this, please do!)

User account and project management

From an operations perspective, the big thing for this infrastructure is that it is low overhead for us. We’d like to empower our community without overburdening us. A big part of this is not having to deal with user accounts, authentication, and authorization. Here’s how I plan on solving this problem:

User accounts are created and maintained by MediaWiki. MediaWiki will use the LDAP Authentication and OpenStackManager extensions. Wiki admins will be able to create accounts for other people. When the account is created, it’ll automatically get an account on the VMs and in OpenStack, as the account will be created in LDAP, and everything in the infrastructure, excluding project wikis, will use LDAP for user accounts. They will also be added to a shared default group, which will give them non-root access to the default project. root access will be granted on an as-needed basis by the operations team in the default project. In this environment, they’ll be able to participate in any default-group shared projects. They will not be able to create VMs.

OpenStack has a concept of “projects”. When a user is added to a project, they have the ability to create, delete, and manage VMs in that project. The default project will be maintained by the operations team. It will be a clone of the production environment. Access to this OpenStack project will be limited. Instead, we’d like real projects (like Usability Initiative, or Pending Changes, or OWA) to have OpenStack projects created specific to the real project. In this project they can create VMs, and maintain separate infrastructure from the default project.

OWA is a good example of when VMs would need to be created for a project. OWA needs LVS, Squid, Apache, MySQL, and a few other things. It runs differently than our current infrastructure, and could require changes that could possibly break other things. For this, the initial configuration could be totally done within a separate set of VMs, where once configurations stabilize they get moved into the default project.

Project management will be handled via the OpenStackManager extension. Anyone in a project will be able to add/remove users to/from the project. Each OpenStack project is also a MediaWiki namespace, and a posix group on all VMs. Though everyone will have access to the default project VMs, only people added to other projects will be allowed to access the VMs in those projects. Users in non-default projects will also have sudo rights automatically granted to them on those VMs.

Access to VMs will be limited to SSH key authentication. Users will be able to manage their own SSH keys via the OpenStackManager extension. These keys will be managed in LDAP, and VMs will sync the keys to the user’s authorized_keys file.

VM, DNS, and Puppet management

The other part of not overburdening the operations team is for VM, DNS, and Puppet management to be mostly hands off. Currently, with the Tesla architecture, I need to create VMs from scratch, configure them, add user accounts and groups, and add users to sudoers. I also need to assign an IP address and add the VMs to DNS. Once a team believes their project is production ready, if there are architecture changes, I need to add those changes to puppet, and recreate the architecture in our production cluster. This is very time consuming. Ideally, most of this can be handled by the developer, and that’s what I’m aiming for with this infrastructure.

When a new project is formed, an operations team member will create an OpenStack project via the OpenStackManager extension, and will add a project member to it. After doing so, that user can add other users to the project, and can create their custom architecture. They’ll be able to go to an interface to create their VMs. When creating the VMs, they’ll be able to name them, give them a size (CPUs, memory, disk space), and manage puppet classes and variables. The puppet configuration will allow them to create VMs that are already pre-configured as specific types of VMs. For instance, if you want a VM that is configured like our Application servers, you’ll simply need to add the “appserver” class.

Once the VM is created, the OpenStackManager extension will add the DNS information and Puppet information to LDAP. When the VM is finished building, it’ll automatically sync user SSH keys from LDAP, configure itself using puppet, and will be available for SSH login.

Everything we do in the production cluster now occurs through puppet, and we’d like developers to do the same thing on their VMs. Though the OpenStackManager extension will only allow selection of configured classes and variables, that list will be managed in the puppet configuration that will be managed through SVN or git. Developers can create puppet manifests, files and templates, and can add them to the repository. The operations team will maintain the main branch, and will merge changes in. When it is time to move a project to the production cluster, we should be able to merge that puppet configuration into our production puppet repository, allowing developers to be part of the process from beginning to end.

Wiki management

We have a pretty annoying problem on Tesla right now. Most projects are sharing the Prototype VM. Some projects use the trunk version of MediaWiki, others use the deployment branch. Most projects share the same extensions directory. This causes problems where projects often break each other. Also, the Prototype VM isn’t configured like the cluster, and as such, code deployed from this environment may run into unexpected issues. A goal of this new infrastructure will be to solve this problem.

Ideally, most projects won’t need to create their own infrastructure for testing. Most projects are just creating MediaWiki extensions that should run perfectly in our existing infrastructure. What developers really need in this situation is a wiki with a full copy of the extensions directory. They need to be able to create a wiki with a choice of either using the deployment branch, or trunk. They need to be able to limit access to their wikis to people in their project, if it isn’t a shared project. They should be able to create these wikis without root access. They need this wiki to run in an environment that is configured as closely to the production cluster as possible.

My plan for this is a script on the default-project, that will run via sudo, that will automatically check out from SVN, create the wiki’s database, and add the wiki to the Apache configuration automatically. It’ll set up the file permissions automatically for a shared-wiki, or a project wiki. Access to these wikis will be controlled via OpenStack projects, which are also posix groups. This gives each project a little flexibility too, as if they later decide they do need a VM for testing, they’ll be able to create it.

Want to help? Have an idea to make this better?

I’m still fairly early on in the process. I’ve built a lot of the infrastructure, and am mostly done with the OpenStackManager extension, but the infrastructure hasn’t launched, and things can still be easily changed. If you want to help, or have ideas on how this can be done better, I’d love to get some help. If you’d like to help with the operations portion of this, I’d love help with that too. It’s a great learning experience, and I’m testing and developing this in the Tesla environment right now, so I can give out access fairly liberally. Even excluding the virtualization part of the infrastructure, we’ll be building a clone of our production environment mostly from scratch, which will be a great learning experience. It isn’t often you get to build something like this from scratch, so if you are interested let me know!

Timeframe

I’m hoping to have something that is at least ready for basic use by mid or late January. Full implementation of my above plan will likely take a few months though. With help I can probably get it fully ready much sooner.

<#comment hash="f92e3f4a596ee1383542fa82e3050512" /> <#comment hash="9d6ee31bc358db3224830f8469fa13c0" />

The New York Times calls North Jersey's financial data centers "the new heart of Wall Street." Here's a glimpse inside these high-tech facilities.

Information Security has easily been too fast of a field to provide reliable predictions. Sometimes it is hard to predict what you find if you come back from a long lunch. But lets try and play along with new years predictions. What will matter to your job this coming year?
We got a running list of various ideas from SANS Instructors [1]. Let me point out two that are sort of my personal favorites:
IPv6: Who would have guessed :) ... I think IANA may run out of IPv4 space sometime this or next week and regional registrars sometime this year. We will keep pushing IPv4 space to the limit and ignore IPv6 for as long as possible. But as usual with procrastination: What we will end up with is a lot of rushed out and broken implementations.
Social Malware: I think we will see less bots that spread via exploits but instead we will see smarter bots that find the right context to trick the user into executing them. Some of it we have seen with bots like Koobface. But there will be more, smarter, versions. Something that assembles an e-mail based on your browser history or facebook groups / pages you like to make it match your interest. You just went to see Tron in the theater? You will get an e-mail or facebook message with a secret second ending as a video file to play. Kind of like spear phishing, but more automated.
Now if you follow what I am doing, you may expect application security as one of the topics. I will skip application security prediction for 2011. I think progress will be incremental and that will be ok. People make plenty of money with secure enough software. There isn't currently a big change that I see coming in 2011. New software will be incrementally better as more developers figure out how to use new tools right. But legacy code will still be a huge problem and it will not be fixed in any big new ways, just one line at a time.
Wikileaks, Cyberwar, Cyber Terror: No big shifts here. It will continue to happen just like in 2010. No big new defenses either. Maybe a bit more international collaboration in fighting malicious actors.
Please feel free to add your predictions as comments below.
[1] http://www.sans.edu/resources/securitylab/security_predict2011.php
------

Johannes B. Ullrich, Ph.D.

SANS Technology Institute

Twitter (c) SANS Internet Storm Center. http://isc.sans.org Creative Commons Attribution-Noncommercial 3.0 United States License.

My wife decided that she wanted to display a list of her latest AudioBoos in the sidebar of her blog. She looked at the AudioBoo JavaScript widget but decided it wasn't subtle enough and so she enlisted me to solve her problem.

It turns out that AudioBoo has an RSS feed, so a simple plugin was required. I had a quick look on the extension site, but most are now "widgets" which her theme isn't set up for or didn't provide an unsigned list. Hence, I whipped up a small extension for her.

It turns out that WordPress ships with MagpieRSS baked in, so the work to get the feed is trivial:

include_once(ABSPATH . WPINC . '/rss.php');
$messages = fetch_rss($url);

The rest of the work is simply formatting the output. The key requirement that she had was that it should provide an unsigned list with title, date and optionally the summary. Along with providing some customisation for her, this is what I came up with:

function akrabat_simple_rss($options = array()) 
{
    $defaults = array(
        'url' => '', 
        'number_of_items' => 5,
        'display_date' => true,
        'date_format' => 'd M Y at H:i', 
        'display_summary' => true,
        'number_of_summary_chars' => 100,
        'link_on_title' => true,
        'link_on_date' => false,
        'css_class' => 'akrabat-simple-rss',
    );
    
    extract (array_merge($defaults, $options));
    
    $output = '';
    if (!empty($url)) {
        include_once(ABSPATH . WPINC . '/rss.php');
        $messages = fetch_rss($url);
        if(count($messages->items) == 0){
            return '';
        }
        
        if($number_of_items > count($messages->items)) {
            $number_of_items = count($messages->items);
        }
    
        $output = '<ul class="'.$css_class.'">';
        for($i = 0; $i < $number_of_items; $i++){
            $message = $messages->items[$i];
        
            $link = $message['link'];
            $title = $message['title'];
            $date = null;
            if (isset($message['published'])) {
                $date = $message['published'];
            }
            if (!$date && isset($message['pubdate'])) {
                $date = $message['pubdate'];
            }
            $summary = null;
            if (isset($message['summary'])) {
                $summary = $message['summary'];
            }
            if (!$summary && isset($message['description'])) {
                $summary = $message['description'];
            }
            
            $output .= "<li>";
            $title_string = htmlspecialchars($title_string);
            if ($link_on_title) {
                $output .= '<a href="http://akrabat.com/'.$link.'">'.$title_string.'</a>';
            }
            $output .= '<div class="akrabat-rss-title">'.$title_string.'</div>';
            
            if ($date && $display_date) {
                $dateString = date($date_format, strtotime($date));
                if ($link_on_date) {
                    $dateString = '<a href="http://akrabat.com/'.$link.'">'.$dateString.'</a>';
                }
                $output .= '<div class="date">'.$dateString.'</div>';
            }
            if ($summary && $display_summary) {
                $summary_string = substr($summary, 0, $number_of_summary_chars);
                if (count(summary) > $number_of_summary_chars) {
                    $summary_string = substr(summary_string, 0, -3) . '...';
                }
                $summary_string = htmlspecialchars($summary_string);
                $output .= '<div class="summary">'.$summary_string.'</div>';
            }
            $output .= "</li>";
        }
        $output .= "</ul>";
    }
        
    return $output;
}

Maybe it's useful to someone else too, and I've documented it somewhere!

I have always been afraid of banks.
– Andrew Jackson

Spot the Vuln uses code snippets from open source applications to demonstrate vulnerabilities in real world web applications. Every Monday morning a vulnerable code snippet is posted. Take a look at the vulnerable code and try to identify where the security vulnerability is. Every Friday, a solution is posted so you can check your answers. Each exercise is designed to last between 5 and 10 minutes. Do it while you drink your morning coffee and you will be on your way to writing more secure applications.

...snip...
<?php

	($hook = get_hook('li_forgot_pass_end')) ? eval($hook) : null;

	$tpl_temp = forum_trim(ob_get_contents());
	$tpl_main = str_replace('<!-- forum_main -->', $tpl_temp, $tpl_main);
	ob_end_clean();
	// END SUBST - <!-- forum_main -->

	require FORUM_ROOT.'footer.php';
}

if (!$forum_user['is_guest'])
	header('Location: '.forum_link($forum_url['index']));

// Setup form
$forum_page['group_count'] = $forum_page['item_count'] = $forum_page['fld_count'] = 0;
$forum_page['form_action'] = forum_link($forum_url['login']);

$forum_page['hidden_fields'] = array(
	'form_sent'		=> '<input type="hidden" name="form_sent" value="1" />',
	'redirect_url'	=> '<input type="hidden" name="redirect_url" value="'.forum_htmlencode($forum_user['prev_url']).'" />',
	'csrf_token'	=> '<input type="hidden" name="csrf_token" value="'.generate_form_token($forum_page['form_action']).'" />'
);

// Setup breadcrumbs
$forum_page['crumbs'] = array(
	array($forum_config['o_board_title'], forum_link($forum_url['index'])),
	array(sprintf($lang_login['Login info'], $forum_config['o_board_title']), forum_link($forum_url['login']))
);

($hook = get_hook('li_login_pre_header_load')) ? eval($hook) : null;

define('FORUM_PAGE', 'login');
require FORUM_ROOT.'header.php';

// START SUBST - <!-- forum_main -->
ob_start();

($hook = get_hook('li_login_output_start')) ? eval($hook) : null;

?>
	<div class="main-head">
		<h2 class="hn"><span><?php echo sprintf($lang_login['Login info'], $forum_config['o_board_title']) ?></span></h2>
	</div>
	<div class="main-content main-frm">
		<div class="content-head">
			<p class="hn"><?php printf($lang_login['Login options'], '<a href="'.forum_link($forum_url['register']).'">'.$lang_login['register'].'</a>', '<a href="'.forum_link($forum_url['request_password']).'">'.$lang_login['Obtain pass'].'</a>') ?></p>
		</div>
<?php

	// If there were any errors, show them
	if (!empty($errors))
	{
		$forum_page['errors'] = array();
	foreach ($errors as $cur_error)
			$forum_page['errors'][] = '<li class="warn"><span>'.$cur_error.'</span></li>';

		($hook = get_hook('li_pre_login_errors')) ? eval($hook) : null;

?>
		<div class="ct-box error-box">
			<h2 class="warn hn"><?php echo $lang_login['Login errors'] ?></h2>
			<ul class="error-list">
				<?php echo implode("ntttt", $forum_page['errors'])."n" ?>
			</ul>
		</div>
<?php

	}

?>
		<div id="req-msg" class="req-warn ct-box error-box">
			<p class="important"><?php printf($lang_common['Required warn'], '<em>'.$lang_common['Required'].'</em>') ?></p>
		</div>
		<form id="afocus" class="frm-form" method="post" accept-charset="utf-8" action="<?php echo $forum_page['form_action'] ?>">
			<div class="hidden">
				<?php echo implode("ntttt", $forum_page['hidden_fields'])."n" ?>
			</div>
<?php ($hook = get_hook('li_login_pre_login_group')) ? eval($hook) : null; ?>
			<div class="frm-group group<?php echo ++$forum_page['group_count'] ?>">
<?php ($hook = get_hook('li_login_pre_username')) ? eval($hook) : null; ?>
				<div class="sf-set set<?php echo ++$forum_page['item_count'] ?>">
					<div class="sf-box text required">
						<label for="fld<?php echo ++$forum_page['fld_count'] ?>"><span><?php echo $lang_login['Username'] ?> <em><?php echo $lang_common['Required'] ?></em></span></label><br />
						<span class="fld-input"><input type="text" id="fld<?php echo $forum_page['fld_count'] ?>" name="req_username" value="<?php echo isset($_POST['req_username']) ? forum_htmlencode($_POST['req_username']) : '' ?>" size="35" maxlength="25" /></span>
					</div>
				</div>
<?php ($hook = get_hook('li_login_pre_pass')) ? eval($hook) : null; ?>
				<div class="sf-set set<?php echo ++$forum_page['item_count'] ?>">
					<div class="sf-box text required">
						<label for="fld<?php echo ++$forum_page['fld_count'] ?>"><span><?php echo $lang_login['Password'] ?> <em><?php echo $lang_common['Required'] ?></em></span></label><br />
						<span class="fld-input"><input type="password" id="fld<?php echo $forum_page['fld_count'] ?>" name="req_password" value="<?php echo isset($_POST['req_password']) ? ($_POST['req_password']) : '' ?>" size="35" /></span>
					</div>
				</div>
<?php ($hook = get_hook('li_login_pre_remember_me_checkbox')) ? eval($hook) : null; ?>
				<div class="sf-set set<?php echo ++$forum_page['item_count'] ?>">
					<div class="sf-box checkbox">
						<span class="fld-input"><input type="checkbox" id="fld<?php echo ++$forum_page['fld_count'] ?>" name="save_pass" value="1" /></span>
						<label for="fld<?php echo $forum_page['fld_count'] ?>"><span><?php echo $lang_login['Remember me'] ?></span> <?php echo $lang_login['Persistent login'] ?></label>
					</div>
				</div>
<?php ($hook = get_hook('li_login_pre_group_end')) ? eval($hook) : null; ?>
			</div>
<?php ($hook = get_hook('li_login_group_end')) ? eval($hook) : null; ?>
			<div class="frm-buttons">
				<span class="submit"><input type="submit" name="login" value="<?php echo $lang_login['Login'] ?>" /></span>
			</div>
		</form>
	</div>
<?php

($hook = get_hook('li_end')) ? eval($hook) : null;

$tpl_temp = forum_trim(ob_get_contents());
$tpl_main = str_replace('<!-- forum_main -->', $tpl_temp, $tpl_main);
ob_end_clean();
// END SUBST - <!-- forum_main -->

require FORUM_ROOT.'footer.php';

About the Authors:
Brett Hardin and Billy Rios run spotthevuln.com, a website dedicated to helping developers understand secure coding practices. You can find out more about the authors by visiting http://spotthevuln.com/about-spot-the-vuln/

Providers focused on cloud computing were the best performers among public companies in the data center sector in 2010, with content delivery networks (CDNs) also scoring huge gains.

On December 1, 2010, the FTC released a preliminary report on "Protecting Consumer Privacy in an Era of Rapid Change" [pdf]. The part of the report that garnered the greatest immediate attention was the FTC's proposal for a "do not track" mechanism modelled on the successful "do not call" option instituted a few years ago. If you have a phone, you can register your number on a list that prevents marketers from making unsolicited calls. (Of course, that doesn't stop charities, companies you have a relationship with, or politicians.) The FTC proposes a similar principle for your browser, giving consumers the option to prevent companies from tracking their online movements.

Consumer privacy is the motivating factor here. A whole industry has grown around the collection and trading of information about consumers. Indeed, it is part of the promise of the online world that content providers who attract consumers can collect more information about them and use this to improve services. If a media outlet can work out that you prefer reading strategic analyzes of markets to what Britney Spears is up to, surely that will make your experience better.

But if that same outlet can work out how that behavior can translate into identifying advertisers who want to pay more to put ads in front of you, things get murkier. And if those same outlets want to see how you got to their site and where you went afterward, the system starts to slide toward stalking and you might like to know what they are going to do with what is ostensibly your information.

The advertising industry, not surprisingly, is concerned about these developments. Advertisers have already had an increasingly difficult time in online environments planning ad campaigns. In the old, offline world, if an advertiser wanted to place an ad in front of newspaper readers, they could put a single prominently display ad in a broad range of papers. Today, if they do the same thing online, there is no guarantee that even a broad campaign will be seen by all readers. Moreover, because consumers no longer read from cover to cover — they now switch around from site to site and within sites — there is every chance some will see your ad too often. That's bad news not only because advertisers pay for wasted impressions but also because it annoys consumers. It shouldn't surprise us that advertisers might reduce their online budgets in response.

Tracking consumers is a response to this issue. If an outlet can tell whether a consumer reading its site today has already seen an ad from one company, it can make sure another is served up in its place. But even more, if an advertising platform (like Google's DoubleClick) can observe if you saw an ad here on HBR.org, they can ensure a different ad gets shown to you should you click on any of the links here. While that type of inter-outlet tracking isn't happening yet on any scale, it is surely something the industry wants to provide.

So the advertising and media industries have a right to be concerned. But we should also remember that the FTC is not proposing a ban on such tracking. They just want to give consumers the right to easily opt out. Consumers who opt out will still see ads — just not ones that are targeted. So it will be more likely the ads are annoying rather than useful.

This should give consumers pause. The decision to opt out of digital tracking is not straightforward. The "do not call" list prevented annoying calls. The "do not track" list may increase the presence of annoying ads.

And so any "do not track" mechanism should remind advertisers and outlets to deliver on the promise of usefulness in order to keep consumers on the program. The danger is that consumers may click their "opt out" option and never see what they have lost.

Joshua Gans is an economics professor at Melbourne Business School and a visiting scholar at Harvard University.

In 1999, the Institute of Medicine reported that American health care was decidedly dangerous for patients. One in every few hundred was hurt, and one in every few thousand was killed by medical misadventures. The cause was not malfeasant individuals; it was inadequately designed and operated systems of care delivery.

Since then, health care providers have invested in a variety of initiatives aimed at improving safety: Electronic order entry to minimize medication mistakes, "kaizen blitzes" and other improvement projects to "lean out" unwieldy processes, checklists to ensure instruments aren't forgotten inside patients, and countless patient safety conferences.

According to findings published recently in The New England Journal of Medicine, things have not improved. Researchers looked at performance changes in North Carolina hospitals supposedly doing the right things — and therefore assumed to be representative of organizations on the cutting edge in improving quality and safety. Their findings: The needle on patient well-being had moved insignificantly.

The only reasonable explanation for this disparity between effort and outcome is that health care leaders are not investing in the right operational changes to achieve excellence in safety, affordability, and capacity.

This conclusion is based on my 10 years of experience helping providers improve care delivery and the documented fact that some select providers have generated profound, across-the-board benefits — they have eliminated complications like ventilator pneumonia and central-line infections, increased capacity, and reduced their costs.

These disparities in outcome are directly attributable to differences in approach.

Unfortunately, many health care organizations continue to cling to the view that improvement can be achieved by purchasing one-off interventions. Their thinking: If they implement enough best-practice bundles here or there to remove the problem and hire enough outsiders to lead improvement projects, things should get good enough. But the sad reality is while this approach will generate improvements, they will not be significant and sustainable.

Why not?

Providing care is complex work requiring well-integrated involvement of people spanning multiple disciplines. Even seemingly "simple" primary-care visits involve administrative staff, doctors, nurses, medical assistants, and technicians. Managing chronic illness, urgent and intensive care, and surgery involve even larger casts. Not only does the number of people make managing care delivery challenging, so do the interdependencies — what one person does affecting and being affected by what many others do.

In short, system complexity is the essential challenge. As a result, those responsible for managing the delivery of care need a methodical approach for designing systems; they can't manage the individual specialties and expect the whole to come together serendipitously. They also need a methodical approach for making things better — a reliable approach for generating improvement and innovation.

The very best deliberately create end-to-end "service lines," manage them across the disciplinary boundaries that often lead to the fragmentation of care, and train themselves and their colleagues to recognize when something is amiss: sterilized equipment that cannot be located, medications that are easy to confuse, orders which are ambiguous, a patient whose condition is drifting unexpectedly. When they see these abnormalities, they swarm them — both to prevent problems from propagating and to understand why they occurred so their recurrence can be prevented. More often than not there aren't silver-bullet solutions. Rather, the whole of the cumulative effect is far greater than the sum of many adaptations.

What does this mean? Health care leaders have to realize that safety cannot be bought like a new diagnostic tool; it has to be earned by engaging health care professionals broadly in seeing and solving problems and incorporating new learning as part of their daily work. (See the HBR articles "Fixing Health Care from the Inside, Today" and "Fixing Health Care on the Front Lines" for what this looks like in practice.) For most this is a profoundly different approach, which requires a new style of leadership. Those most senior have to lead the charge and cannot delegate the responsibility to a continuous-improvement staff. Hopefully, more will, so when the next study is done, we'll find that health care has improved in quality, access, and affordability.

Steven Spear is a senior lecturer at MIT's Sloan School of Managementand a senior fellow at the Institute for Healthcare Improvement. The author of the book The High Velocity Edge, he also wrote the McKinsey Award-winning HBR article "Fixing Health Care from the Inside, Today" and co-authored "Decoding the DNA of the Toyota Production System."

There's no doubt about it; numbers don't lie. Companies that demonstrate strategic coherence — think Wal-Mart and Coca-Cola — earn a market premium in terms of higher earnings and greater shareholder value. The big question for many leaders as they look toward 2011 is: "How can my company be one of them?"

Strategic coherence results from your ability to connect what you sell (your products and services) with your unique and differentiating capabilities (what you, as a company, do to be great) — all within the framework of a clear way to play (your way of creating value for your customers).

In trying to "be great," many of today's leaders are caught in a trade-off dilemma between focusing on those businesses where the company's unique strengths matter versus achieving the growth that Wall Street seems to want, regardless of whether that growth will come with long-term financial success.

Leaders need to look past Wall Street-pleasing adjacencies or new businesses, and instead make sure their core strategy will result in long-term success. However, that doesn't mean restricting growth; in fact, it opens up new doors for growth in unexpected areas that still leverage your differentiating capabilities.

Our research shows that coherent companies earn what we call a coherence premium because strategic coherence provides greater differentiation, enables the right form of scale, focuses limited investments, and has the hugely important benefit of aligning the entire organization around a common and consistent purpose.

Putting Capabilities at the Foundation of Your Competitive Advantage

First, let's make sure to define capabilities clearly: by capabilities, we mean the interconnected people, knowledge, systems, tools, and processes that create differentiated value for customers. That's because winning strategies don't start outside the company. Competitive advantage stems from what the company does better than any other and from using those capabilities over and over again to create value for customers. The first and most important step toward strategic coherence is identifying those unique sources of value.

To lay the right foundation:

1. Treat strategy, capabilities and cost together: What the company already does best should be driving its strategic direction. Don't choose a strategic direction and then wonder how you can build the required capabilities. Start from the opposite direction: Find an attractive market that values what you do best! Similarly, think about every item of cost as an investment. Disproportionately allocate your costs to your essential capabilities and streamline the remaining areas - this is what propels your company to greatness.
2. Focus on capabilities rather than just fixed assets: Fixed assets, including brands, are more difficult to leverage across diverse businesses and tend to expire, become obsolete, or give way to related services. The competitive value of capabilities, however, will only grow as you apply them to your entire portfolio of products, day in and day out.
3. Identify your differentiating capabilities: Identify what your company does particularly well, what your customers value and your competitors can't beat. Such capabilities could be rapid-cycle product development, point of sale merchandising, large-scale fabrication, and so on. Make sure to sort out which capabilities are merely table stakes in your markets, versus those capabilities that truly differentiate your company and create a competitive advantage.
4. Define your way to play: Be specific about how you're going to approach the market, i.e., your way to play, and base this on what you already do well. Define precisely how your way to play adds value for your chosen customers (e.g., as an innovator, a value player, or an experience provider) and how it differentiates you from your competitors.
5. Integrate capabilities into a system: Develop capabilities that are mutually reinforcing since such capabilities systems provide stronger support for the company's chosen way to play and are almost impossible to copy. Frito Lay's capabilities system, combining direct store delivery, continuous innovation of new products, and a proficiency with local consumer marketing programs that reinforce demand, provides a perfect example.

Earning the right to win is never a cakewalk. But we believe a capabilities-driven strategy is the most direct, efficient, and effective way to get there. With the right capabilities in place - strengthened and refined over time - winning companies are well-positioned for the right kind of growth, and they lead their market by delivering unique value to customers that competitors can't beat.

Paul Leinwand is a Partner in Booz & Company's global consumer, media, and retail practice. He serves as chair of the firm's Knowledge and Marketing Advisory Council. Cesare Mainardi is Managing Director of Booz & Company's North American business and is a member of the firm's Executive Committee. They are co-authors of The Essential Advantage: How to Win with a Capabilities-Driven Strategy, published by Harvard Business Review Press. For more information, visit theessentialadvantage.com.

As retailers tally up sales from the crucial holiday period, the early returns look good: spending rebounded in 2010, with MasterCard reporting a 5.5 percent increase over the previous year.

But those gains haven't been spread equally. Online spending jumped by 15.4 percent — and the 2010 holiday season may mark the first time that online spending accounts for more than 10 percent of all gift purchases. Even before the holidays, brick-and-mortar retailers were suffering: when Best Buy reported disappointing third-quarter results in early December, analysts attributed the poor numbers to increased competition from online retailers — and Best Buy's stock dropped 18 percent in one week.

The proliferation of smart phones and apps like ShopSavvy, which allow mall-goers to easily compare online prices to get the best deal, will only increase the threat to physical stores.

But even as Americans embrace e-commerce, physical retailers still provide a valuable service — and I've been thinking about a new system that would allow them to better compete.

I was reminded of the value offered by brick-and-mortar retailers during my own recent holiday splurge. Every time I entered my local Costco store last year, I was greeted by banks of flat screen televisions blasting vivid cinema-like images. When prices dropped over the holidays, it was time to make my purchase.

I searched around online, and settled on a highly-discounted model from an Internet retailer. Before buying it, I stopped in at Sears to check its selection. A helpful sales associate came by to talk about my choice. I'd focused on brand and pixel clarity as my primary criteria. The associate explained why I should also focus on other attributes such as the refresh rate (important for fast moving images) and LED backlight (which provides brighter display and greater contrast). That brief tutorial made me rethink my purchase — and in retrospect, he saved me from buying a model I'd have regretted.

The televisions at Sears cost more than they do at e-commerce sites. That makes sense: physical retailers have to cover costs (such as pricey mall real estate, store fixtures, cashiers and sales associates) not incurred by Internet retailers. Online stores further benefit as many states do not require them to charge sales tax. In an increasingly tech savvy and price sensitive world, how can physical stores compete?

The conventional wisdom is that consumers will pay a premium for the convenience and service provided by brick and mortar stores. But as online retailers' growing market share attests, that CW isn't holding up. I went online to buy the TV the Sears associate helped me select, and I saved $150. As the sliding value of Best Buy shares confirm, too many shoppers aren't willing to pay a premium for sales associates, in-person demonstrations, or the ability to get a product right now. The current retailing model, which expects consumers to pay this premium, is starting to look broken.

It's time for a new system in which manufacturers help compensate physical retailers for the value they bring to the sales proposition. They can do that by offering brick and mortar retailers lower wholesale prices than their web counterparts. I call this discount the Physical Store Equalizer, or PSE.

Retailers' pitch to manufacturers to try to gain this discount should be straightforward: "As a brick and mortar retailer, we add value and generate higher sales of your product. Our stores increase your brand awareness, provide a venue for people who want to touch and feel the product before they buy it (whether they buy it from us or online), and our sales staff help educate your buyers. We bear costs for these services, so it's impossible for us to match online prices of your product. To be fair to us, we require a wholesale price that is 10% less than what you are offering web retailers."

This discount can vary based on the value-added: products that benefit more from being sold at physical outlets (appliances, for instance) can have a higher discount while those that receive fewer benefits (which include homogenous, less expensive goods on which people aren't as price sensitive). These terms should also apply to "like" products. This prevents manufacturers from circumventing this clause by adding a superfluous attribute or simply changing the model number.

This strategy focuses on a key pricing principle that all B2B and B2C companies need to practice: articulate and charge for the value that you provide. Faced with a Physical Store Equalizer, manufacturers will have to decide whether the value provided by a brick and mortar retailer merits the smaller margins — and the potential backlash from Internet retailers.

As consumers become more comfortable with online shopping and price comparison technology becomes ubiquitous, physical retailers must confront this pricing issue today. Brick and mortar retailers need only look at Borders' recent announcement that it is delaying payments to vendors to see what is potentially on the horizon for them if they don't work to change the existing economic model.

Rafi Mohammed is a pricing strategy consultantand author of The 1% Windfall: How Successful Companies Use Price to Profit and Grow (HarperBusiness).

A couple of days ago, Dimitri published a blog post, Analyzing Percona's TPCC-like Workload on MySQL 5.5, which was  a response to my post, MySQL 5.5.8 and Percona Server: being adaptive. I will refer to Dimitri's article as article [1]. As always, Dimitri has provided a very detailed and thoughtful article, and I strongly recommend reading if you want to understand how InnoDB works. In his post, Dimitri questioned some of my conclusions, so I decided to take a more detailed look at my findings. Let me show you my results.

Article [1] recommends using the innodb_max_dirty_pages_pct and innodb_io_capacity parameters to get stable throughput in MySQL 5.5.8. Let's see what we can do with them. Article [1] also advises that innodb_log_file_size is not important for stable throughput.

For my tests, I again used the Cisco UCS C250 box with 346GB of RAM , and I ran the tpcc-mysql benchmark with 500W (about 50GB of data) on the FusionIO 160GB SLC card. For innodb_buffer_pool_size I used 26GB to represent about a 1/2 ratio of buffer_pool_size to data.

For the initial tests, I used MySQL 5.5.8 (the tar.gz binary from dev.mysql.com), and for the other tests I used Percona Server based on 5.5.8. Addressing a complaint to my previous post, I am sharing the percona-server-5.5.8.tar.gz I used for testing, but please note: It is very pre-beta and should not be used in production. You can download it from our TESTING area.

In order to test different settings in a short period of time, I used 30-minute runs, which may not be long enough to see the long-term trend, but we will see the effects anyway. The full command line to run the test is: tpcc_start localhost tpcc500w root "" 500 32 10 1800. For better understanding the results for each run, I will show different graphs:

• benchmark throughput - This is New Order Transactions per 10 seconds.
• dirty page - This graph will contain the percentage of dirty pages in the InnoDB buffer pool. This value is calculated from the output of mysqladmin ext -i10 using this formula: (100*Innodb_buffer_pool_pages_dirty)/(1+Innodb_buffer_pool_pages_data+Innodb_buffer_pool_pages_free). This is the exact formula that InnoDB uses internally to estimate current innodb_dirty_pages_pct.
• checkpoint age - This is a value in MB or GB and shows what amount of the space in innodb_log_file corresponds to changed pages in the buffer pool. You can compute this value as Log sequence number - Last checkpoint at from SHOW ENGINE INNODB STATUS.

Here are the InnoDB settings for the initial run. Later I will change them in searching for optimal values.

Please note that initially I used the default value for innodb_max_dirty_pages_pct, which is 75, and the default value for innodb_io_capacity, which is 200. I also enabled innodb_doublewrite. As will appear later, it is quite a critical parameter.

So, the results for the initial run, using MySQL 5.5.8:

Let me explain the second graph a little. I put checkpoint age and dirty pages percentage on the same graph to show the relationship between them. Checkpoint age is shown by the red line, using the left Y-axis. Dirty pages are shown by the blue line, using the right Y-axis.

As expected, throughput jumps up and down. Checkpoint age is stable and is about 2854.02 MB. Checkpoint age is the limiting factor here, as InnoDB tries to keep the checkpoint age within 3/4 of the limit of the total log size (total size is 2000MB*2).

The 15-minute average throughput is 59922.8 NOTPM.

Okay, now following the advice in article [1], we will try to limit the percentage of dirty pages and increase I/O capacity.
So, I will set innodb_max_dirty_pages_pct=50 and innodb_io_capacity=20000.

Here are the results:

As we see, throughput is getting into better shape, but is far from being a straight line.
If we look at the checkpoint age/dirty pages graph, we see that the dirty pages percentage is not respected, and is getting up to 70%. And again we see the limiting factor is checkpoint age, which is getting up to 3000MB during the run.

The 15-minute average result for this test is 41257.6 NOTPM.

So, it seems we are not getting the stable result of article [1], and the difference is the doublewrite area. Doublewrite activity actually adds significant I/O activity. Basically, it doubles the amount of writes , as you see from its name. So, let's see what result we have when we disable doublewrite; that is, set innodb_doublewrite=0.

Now, although throughput is not a perfect line, we see a totally different picture for dirty pages and checkpoint age.
The dirty page maximum of 50% is still not respected by InnoDB, but the checkpoint age drops far below the 3000MB line. It is now on about the 1500MB line.

The 15-minute average result for this test is 63898.13 NOTPM. That is, by disabling the doublewrite area, we improved the result 1.55x times.

As it seems hard for InnoDB to keep 50% dirty pages, let's try 60%.

Here is the run with innodb_max_dirty_pages_pct=60.

Okay, now we finally see throughput more or less flat. The dirty page percentage is kept at the 60% level, and checkpoint age is at the 2000MB level; that is, not bounded by innodb_log_file_size.

The 15-minute average result for this test is 64501.33 NOTPM.

But we still have DOUBLEWRITE=OFF.

Since now we are limited by innodb_max_dirty_pages_pct, what will be the result if we try to increase it to 70% ?

It seems 70% is too big, and now we again hit the limit set by innodb_log_file_size.

The 15-minute average result for this test is 57620.6 NOTPM.

Let me summarize so far. With innodb_doublewrite disabled, we have stable throughput only with innodb_max_dirty_pages_pct=60. Setting this value to 50 or 70 gives us dips in throughput, though for different reasons. In the first case, InnoDB is unable to maintain the 50% level; in the second we are limited by the capacity of REDO logs.

So, what do we get if we again enable innodb_doublewrite, but we now set innodb_max_dirty_pages_pct=60?

This is a bummer. Throughput again jumps up and down. The dirty pages percentage is not respected, and InnoDB is not able to maintain it. And checkpoint age is back to 3000MB and again limited by innodb_log_file_size.

The 15-minute average result is 37509.73 NOTPM.

Okay, so what if we try an even smaller innodb_max_dirty_pages_pct, setting it to 30? (I use a 1-hour run in this case.)

The results:

I can't say if the resullt should be considered stable. There are still a lot of variations.

The 15-minute average result is 37039.73 NOTPM.

Let's try an even larger decrease, setting innodb_max_dirty_pages_pct=15.

This seems to be the most stable line I can get with MySQL 5.5.8.

The 15-minute average result is 37235.06 NOTPM.

This allows me to draw a conclusion which partially concurs with the conclusion in article [1]. My conclusion is: With doublewrite enabled, you can get a more or less stable line in MySQL 5.5.8 by tuning innodb_max_dirty_pages_pct and innodb_io_capacity; but the limiting factor is still innodb_log_file_size.

To prove it, I took Percona Server based on 5.5.8 and ran it in MySQL mode (that is, using adaptive_flushing from InnoDB and with the adaptive_checkpoint algorithm disabled), but with giant log files. I used a log file of 8000MB*2, just to see what the maximum checkpoint age is.

Okay, here are the results:

Success! With a big log file, we are getting stable throughput. Checkpoint age jumps up to 3900MB line, but the dirty page percentage is not kept within the 60% line, going instead up to the 70% line limit. That is, to get this stable throughput, we need a total log file size of about 3900MB + 25% = 5300MB.

The 15-minute average result for this test is 48983 NOTPM.

But what about innodb_max_dirty_pages_pct; can we get better results if we increase it? It's not respected anyway.
Let's try the previous run, but with innodb_max_dirty_pages_pct=75.

The 75% dirty pages line is at a stable level now, but something happened with throughput. It doesn't have holes, but there is still oscillating. Checkpoint age is quite significant, reaching 7000MB in the stable area, meaning you need
about 9000MB of log space.

The 15-minute average result for this test is 55073.06 NOTPM.

What can be the reason? Let's try a guess: flushing neighborhood pages.
Let's repeat the last run, but with innodb_flush_neighbor_pages=0.

Okay, we are back to a stable level. Checkpoint age is also back to 3000MB, and dirty pages are stable as well, but getting to 77%. I am not sure why it is more than 75%. It is a point for further research, but you are probably tired from all these graphs, as am I.

The 15-minute average result for this test is 52679.93 NOTPM. This is 1.4x better than we have with the stable line in MySQL 5.5.8.

But, finally, let me show the result I got running Percona Server in optimized mode:

The 15-minute average result is 73529.73 NOTPM.

The throughput is about 1.33x better than in "MySQL compatible mode", though it requires 10500MB for checkpoint age; that is, 14000MB of log space. And, the Percona Server result is ~2x better than the best result I received with MySQL 5.5.8 (with innodb_doublewrite enabled).

In summary, my conclusion is: You can try to get stable throughput in MySQL 5.5.8 by playing with innodb_max_dirty_pages_pct and innodb_io_capacity and having innodb_doublewrite enabled. But you must have the support of big log files (>4GB) to help increase throughput.

Basically, by lowering innodb_max_dirty_pages_pct, you are killing your throughput. When you disable innodb_doublewrite, you can get stable throughput if you are lucky enough to find a magic innodb_max_dirty_pages_pct value. As you saw in the results above, 50 and 70 are not good enough, and only 60 gives stable throughput.

(Post edited by Fred Linhoss)

---

Entry posted by Vadim | 10 comments

Add to: | | | |

In my post MySQL 5.5.8 and Percona Server: being adaptive I mentioned that I used innodb-log-block-size=4096 in Percona Server to get better throughput, but later Dimitri in his article MySQL Performance: Analyzing Percona's TPCC-like Workload on MySQL 5.5 sounded doubt that it really makes sense. Here us quote from his article:

"Question: what is a potential impact on buffered 7MB/sec writes if we'll use 4K or 512 bytes block size to write to the buffer?.. )
There will be near no or no impact at all as all writes are managed by the filesystem, and filesystem will use its own block size.. - Of course the things may change if "innodb_flush_log_at_trx_commit=1" will be used, but it was not a case for the presented tests.."

Well, sure you do not need to believe me, you should demand for real numbers. So I have number to show you.

I took Dell PowerEdge R900 server with 32GB of RAM and with FusionIO 320GB MLC card, and run tpcc-mysql benchmark with 500W using Percona Server 5.5.8.

Here is relevant part of config what I used

I made two runs, one with default innodb-log-block-size ( 512 bytes), and another with --innodb-log-block-size=4096. Full benchmark command is tpcc_start localhost tpcc500 root "" 500 24 10 3600

From graph you can actually see, that there is quite significant impact when we use --innodb-log-block-size=4096.

The average throughput for last 15 mins in first run is 38090.66 NOTPM,
in second run it is 49130.13 NOTPM, that is increase is 1.28x, and I can't say this is "near no or no impact".

What is the cause of such difference ? I am not really sure. Apparently FusionIO driver is sensitive to IO block size. And I know that other SSD/Flash drives like to have IO multiplied to their internal block size (which is often 4096 bytes), but I do not know if the effect is the same as on FusionIO.

I put CPU usage graph ( user and system) for both cases:

You may see with 4096 block size USER and SYS CPU is utilized much better, meaning that IDLE is much lower.
Is this contention issue in FusionIO driver when we have 512 bytes IO ? It may be.

Also I am not sure what is strange hill on throughput line with 512 bytes, but it is quite repeatable.
My blind guess (but do not believe me, I have no proof) is that again something is going on inside FusionIO driver,
but this is topic for another research.

For history, FusionIO card information is

---

Entry posted by Vadim | 15 comments

Add to: | | | |

I’m speaking at MySQL/PHP Meetup at Charlotte,NC next week with topic being Scaling MySQL Applications. We will have a presentation probably for about an hour and the good amount of time for questions. So come prepared.

Great thanks to Red Ventures, LLC for help organizing and sponsoring this event. I’m hearing there are going to be a door prizes, some snacks and drinks available on this event.

---

Entry posted by Peter Zaitsev | No comment

Add to: | | | |

How can we improve how organizations learn and adapt in fast-moving industries? I recently spoke to a group of Silicon Valley HR leaders and this question came up, along with the related matter of how to do knowledge management better. They asked how we do knowledge management at frog design, since we have studios spread all over the world and work in a diverse range of industries. Frog does relatively little formal knowledge management, but as we talked I realized that there are many things we do to support learning but don't get recognized as knowledge management. I suspect the same is true at many organizations.

Many of the approaches at frog are quite casual in nature, and focus more on preparing for the future than they do on codifying the past — an essential factor in industries such as tech and mobile, where knowledge becomes outdated scant months after it is acquired. I dubbed this "pre-emptive knowledge management." It assumes that the future will be different from the past, and therefore generates and communicates knowledge without a clear sense of how it will be useful later. Obviously this creates surplus knowledge which never becomes relevant to you. But, if handled right, it also has a better chance of creating knowledge that will help inform the next chapter of your business.

I clustered a variety of KM approaches into a 2x2, and I'll describe some of the specific things we do at frog by way of illustration. This is not to say that these are the best methods, but they do show that there are often many below-the-radar and informal methods of achieving knowledge sharing that don't get recognized as being valuable for knowledge management.

Formal/Historical Methods

Intranet: Like many companies, we have an intranet, and this is where groups of domain experts across disciplines and locations can share knowledge in the form of presentations, whitepapers, and case studies. We have a number of expert groups, cutting horizontally across the organization, that serve as focal points for building knowledge in specific areas, such as healthcare or energy.

Employee Directory: Employees can self-identify their expertise and interest areas, making it possible to find out who knows what about what, without having to know someone's name ahead of time.

Resourcing Software: In a similar vein, we custom-built a piece of software for managing resources across studios, and that automates historical tracking of which projects and clients someone has worked on.

Retrospectives: Quick "field reviews" of lessons learned while a project is still in progress, so it's not too late to put the learning to work (as opposed to postmortems).

Formal/Pre-emptive Methods

We have a variety of means to capture forward-looking knowledge in a structured way:

Blogs: We have almost 40 staff-written blogs.

Magazine: frog produces a print magazine several times a year with articles by external experts and frog staff.

Conferences: frog staffers participate in over 200 conferences and workshops each year.

Collectively these forums generate a large amount of knowledge throughout a year, some of which has no obvious connection to what we as a company do at all — at least not yet.

That's the key to pre-emptive KM: Always look ahead and be curious about the world around you. You will acquire knowledge before its relevance becomes apparent. Even if you don't use the knowledge specifically, often it can lead to unexpected inspiration or shifted perspectives.

Ad Hoc/Historical Methods

This involves providing knowledge as it comes up and doing so in a loose manner, or soliciting expertise on an as-needed, just-in-time basis.

Project Presentations: Teams regularly present recently completed projects to the weekly all-hands meeting in each studio, in the process sharing knowledge about new technologies, methods, user insights, trends with the local group. (These presentations are always done within the auspices of our confidentiality agreements, which may delay some work getting shown, or perhaps not shown at all.)

Email Blasts: People will send out email blasts when they need expertise on an unfamiliar topic. These are just-in-time emails, sent only when up-to-the-minute knowledge is required.

Ad Hoc/Pre-Emptive Methods

We use a variety of habits to encourage as much mutual knowledge sharing and inspiration as possible.

Open Plan Seating: We have no cubicles, the studios are all open plan, and different disciplines (design, engineering, strategy, etc.) are mixed together. That layout maximizes inspiration just by walking around and seeing different approaches and ideas jammed up against one another. It also helps build a social fabric and sharing of ideas and methods between the different disciplines who could otherwise easily become silos.

Move Staff Between Studios: Staff frequently spend anywhere from two weeks to two months in studios in other cities or countries. This again allows sharing of methods and ideas, and gaining insights from diverse cultures.

Life Stories: Employees can give short presentations to the all-hands meetings about their lives, blending talking about personal history, hobbies, family, and work experience. While these are historical, really they help with pre-emptive knowledge management by bringing to the surface people's hidden talents, experiences and interests that might otherwise go undiscovered and under-utilized.

I'm sure there are lots of other knowledge-enabling tricks that don't get proper recognition. I'd love to hear comments from others' about their experiments with managing knowledge and improving learning in fast-paced companies.

Adam Richardson, a Creative Director at global innovation firm frog design, is the author of Innovation X: Why a Company's Toughest Problems are its Greatest Advantage. He can be found on Twitter at @Richardsona.

There is something about a big snowstorm that brings out the best, or more often the worst, in big city mayors. If, as former Speaker of the House Tip O'Neill once said, "All politics is local," then you would think that the first hint of snow in the forecast would prompt mayors to relocate their offices temporarily to where the snowplows are dispatched.

Mayor Mike Bloomberg, who recently won a third term, and is widely regarded as an adept city leader, has come under fire for inefficient snow removal in New York City. Most of the complaints came from residents in the so-called outer boroughs — e.g., the four that are not Manhattan.

But across the Hudson, Newark Mayor Cory Booker has received acclaim for his response to the nor'easter snowfall, despite taking flak on a host of other matters. Not content with supervising removal, he plunged in with a shovel, helping to extricate cars, clear walkways, and in one instance deliver diapers to a housebound mother. He also tweeted his first-hand observations of the snow to his more than one million Twitter followers.

We like to see our elected officials in action. The contrast between Bloomberg's reception and Booker's can serve as a lesson for anyone in a position of authority. Here are some tips for the next big storm that hits your office:

Take a moment to figure out what's going on.An executive I know experienced a major disruption in service to his company. He was the person in charge and he told me that at the first response meeting everyone started talking at once. The chatter was nervous response — not constructive — so he delegated responsibilities and then called for a subsequent meeting in an hour's time. This also helped to impose order on a chaotic situation.

Act promptly, not hurriedly. A leader must provide direction and respond to the situation in a timely fashion. But acting hurriedly only makes people nervous. You can act with deliberateness as well as speed. Or as legendary coach John Wooden advised, "Be quick but don't hurry."

Manage expectations.When trouble strikes, people want it to be over right now — but seldom is this kind of quick resolution possible. It falls to the leader in charge to address the size and scope of the crisis. You don't want to alarm people, yet do not be afraid to speak to the magnitude of the situation. Winston Churchill was a master at summing up challenges but offering a response at the same time. As he famously said when taking office in 1940, "You ask, what is our aim? I can answer in one word. It is victory; victory at all costs; victory in spite of all terror; victory, however long and hard the road may be, for without victory there is no survival."

Demonstrate control. When things are happening quickly, no one may have control, but a leader can assume control. That is, you do not control the disaster — be it man-made or natural — but you can control the response. A leader puts himself into the action and brings the people and resources to bear. Think of Red Adair, who made a name for himself putting out oil fires that no one else could. A raging blaze may seem uncontrollable but Adair knew could control the way it was extinguished.

Keep loose. Not only does this apply to personal demeanor — a leader can never afford to lose composure — it applies to the leader's ability to adapt rapidly. A hallmark of a crisis is its ability to change quickly; your first response may not be your final response. In these situations, a leader cannot be wedded to a single strategy. She must continue to take in new information, listen carefully and consult with the frontline experts who know what's happening.

As much as we like to see senior executives pitch in and help with the heavy lifting, there is a limit. A senior executive's prime role is setting direction. If he or she is engaged too much in front line responsibility, then who is doing the vision thing? Some executives still enjoy doing that hands-on work; they like the rush of adrenaline that comes from direct action. Too bad. That is not their job any more.

Leaders have another important role during a crisis and that is to provide perspective. As Mike Useem has written in The Go Point, an insightful study of decision-making, effective leaders can often do more by standing back from the action.

It is why, as Useem notes, that the team leader in mountaineering expeditions often remains at base camp rather than hiking to the summit. That way, if trouble strikes, he can direct the response with the perspective that comes from seeing the mountain as a whole and the conditions that affect the summit team.

The measure of a leader is often tested during a crisis. And those leaders who can engage directly, but still maintain their sense of perspective, are the ones that will help the organization survive.

John Baldoni is a leadership consultant, coach, and speaker. He is the author of nine books, including 12 Steps to Power Presence: How to Assert Your Authority to Lead. See his archived blog for hbr.org here.

The final set of MCM training videos we recorded for Microsoft have been posted on TechNet and are available for download.

For your convenience, I've created a page that lists all the videos, with links, grouped together to match thetraining classes we provide.

These videos are designed to give you an idea of the breadth of knowledge required for the certification, but they're also great primers for our classes. The material covered in the videos will be covered during the classes, but in much more depth, with a lot more additional material, and with different demos.

Check out the videosHERE.

Enjoy!

PS Some people have asked for the demo scripts from the videos - these will be blogged by us through the year.

One of the key features of any web application is the ability to gather input data from the user. HTML Forms have been the way to do that and over the years, developers have innovated to bring a better experience...

Have you ever wondered why there is a $ sign in the JavaScript code used to extend jQuery? We have the answer.

Its been an amazing five years but finally the servers are powering down on the backstage.bbc.co.uk (to be official) project.The legacy of backstage will live on, but what better way to end the project but to launch a ebook which tells some of the stories of how the project started way back in 2004.

We commisioned Suw Charman-Anderson to create the eBook retrospective of the whole project, quite a challenge as you can imagine. But she's done a excellent job with help from editor Jim McClennan and designer Nicola Rowlands. Its an fantastic piece of work I think you will agree.It also serves as a very fitting tribute to the endless efforts of the many staff, friends, hackers, developers, designers, critics, etc, etc of the project over the last half decade.

There is plenty of background information in the ebook including those playground servers, the amazing array of prototypes and some real interesting points about the nature hacking... maybe someone should update wikipedia with some of the information?

Ladies and Gentlemen, I'm really happy to present the Backstage Ebook:Hacking the BBC, a backstage retrospective. From us to you, for making backstage what it became.

Download in [PDF] [print ready PDF] [EPUB] [MOBI] [RTF]

(The Amazon Kindle will read MOBI files, the Apple iPad makes great work of the PDF, while most others readers will accept epub. If you're in any doubt try the PDF on which is full colour or the RTF which is just the content.)

You can also read more on the Guardian's PDA blog,  the BBC R&D blog (which will be the place to find out what happens post backstage), the BBC internet blogand of course my own personal blog.

I think its safe to say the ebook is licenced under a Creative Commons BY-NC-SA licence. So please do share it and tells us what bits you loved or hated. In actual fact, tells us what bits you loved or hated generally about the whole project. We really value your feedback and of course everything you all did to make backstage your place to influence the BBC as a whole.

thank's for all the tags

.... end line :)

Back in October 2010 the BBC announced that BBC Backstage - the developer platform and open data project I had created with Tom Loosemore and James Boardwell back in 2004 - would be closing at the end of the year.

It was sad news, but one that was both expected and appropriate. The project set out to do big things:

• introduce a large and buerocratic media organization to the concepts of open data,
• share that data with 3rd party developers in order to let them find new and experimental uses for it
• foster internal and external innovation practices that were new, chaotic and sometimes challenging to an old encumbant.

But I think its fair to say that on the whole, the project met its goals and expectations.

As a by-product I think BBC Backstage, and the community that formed around it, also helped kick-start the fledgling London Startup community that we have today. What was then called “The London New Media Scene”, primarily because of the agency orientated slant of the London industry at the time, influenced a generation of non-commercial hackers and NTK subscribers to become entrepenurial and start building startups.

With BBC Backstage winding up, the BBC has produced a wonderful retrospective, “Hacking the BBC”, which I had the honour of being interviewed for. You can download a copy here (pdf) or see below.

The closure of BBC Backstage is certainly a sad day for me, but at the same time I’m confident that it was time to do it. The challenge for the BBC is maintaining the concept of open data and external innovation – and weaving it through the entire fabric of the organization. They claim that is something that is happening, and I think there are good people there championing the notion – but I think the BBC still has some way to go before that box can be really ticked.

You can read Jemima Kiss’s coverage on the Guardian’s website or you can check out a few photo memories I have of the project:

A very flush-faced looking me launching the project at OpenTech 2005 (photo by Natalie Downe)

The BBC Backstage Team winning a New Statesman Award for innovation, 2006

and of course, cheekily snapping Tom Loosemore in a suit:

“Have you ever noticed how certain questions come up again and again on Stack Overflow sites?”

— From The Wikipedia of Long Tail Programming Questions, over on the Stack Overflow blog.

Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.

Over the holidays, I used some of the vacation and down time to reorganize my home network. Part of this was to update my network maps and figure out how many of my devices do not support IPv6. Ido use IPv6 extensively at home, but even some recently purchased devices do not support it.
Another problem you have with IPv6 is to find all devices on your network. The standard and simplest way to do this (aside from passively listening)is to ping the all hosts multicast address ff02::1. If you use auto configured link local addresses, you can also look for the EUI-64 (MAC Address) derived IPv6 addresses.
The result: ashell script to run some of these scans for you [1]
The ipv6finder.sh script currently is tested on Linux and OS X. It will not work on Windows. It does require root access as it uses arping for some of its tests (could fix that, but I found the arping output to be more consistent between platforms then just the arp command which would work too with a normal ping).
Read the comments in the file for some more details. Also: at the top of the script there are some variables that you can use to point it to the right location for various binaries it uses. Why bash and not perl... well, I started it in bash and it grew.
[1] http://johannes.homepc.org/ipv6finder.sh
------

Johannes B. Ullrich, Ph.D.

SANS Technology Institute

Twitter (c) SANS Internet Storm Center. http://isc.sans.org Creative Commons Attribution-Noncommercial 3.0 United States License.

For a test project I was doing I wanted a way to collect data about the device, but since you can't use JavaScript from an Android application created in AS3, because its not hosted within an HTML page, this forced...

Last year saw Linux fight free of one legal morass, and perhaps right into another; Microsoft take another swing at replacing XP; and Apple bring the App Store model to the desktop.

Dave Gunn, technical manager at the Royal National Institute of Blind People and a speaker at TOC 2011, discusses the bright future of accessible publishing and how it offers moral and financial benefits.

So I started playing with Zend_Json_Server and was having a hard time trying to figure out how to call the server from a client. Finally I checked the JSON-RPC 2.0 spec and read a very important detail that I had not realized:

The Request is expressed as a single JSON Object

This was the key to my problem. There are 4 parameters that can be sent with each JSON-RPC 2.0 request but I was sending each as a post var which is not what Zend_Json_Server expects. It simply wants one json encoded object with all the parameters inside. The 4 available parameters are:

1. jsonrpc – the version you are using; I’m using 2.0.
2. method – the name of the method you want to call in the server.
3. params – object of parameters your method needs. If you don’t need any, don’t send this param.
4. id – an identifier (anything you want) that will be sent to and from the server for this request.

I’m using Zend_Http_Client() to make the request and here is an example:

And here is what I see in my browser when I make the call:

Basically, all I had to do was

• create an array of all the parameters I want to send on my request object. I’m calling the find method and passing an id
• create a new client object
• set the url to where your setting up your Zend_Json_Server
• set the client to post (Zend_Json_Server only seems to handle post at this time)
• json_encode your parameters array
• pass the json_encoded object to the setRawData so it passes exactly what we want which is one json object to the server
• call request to send the request and finally get body to show the response.

Here is my code for the server.php file so you can see how I’m setting up the server:

I setup the server instance, add a class to handle the requests, call handle and echo the response. Student will look for the Student.php class so make sure it’s available in your include paths. That’s it, hope you found this helpful. As a side note, I noticed Zend_Amf_Server has a method to addDirectory instead of adding each class separately. Hopefully this gets added to Zend_Json_Server at some point.

Today in the New York Times, Simon Johnson writes a piece entitled "Why Are Taxpayers Subsidizing Facebook, and the Next Bubble?". The central thesis of the piece is whether it is appropriate for an institution like Goldman that is "too big to fail" and therefore effectively has a government guarantee that it will not default on its debt, be an equity investor in Facebook.

It is one thing to invest money on behalf of your clients. Thats the core of what an investment bank does. No problem there. But we heard today that the division of Goldman that does just that, passed on the $450 million dollar investment in Facebook. As a result, Goldman itself did the investment, putting the deal directly on the company's balance sheet.

Simon argues that this is problematic because the Goldman balance sheet is effectively guaranteed by the government, and because Goldman is an actual bank (as opposed to an investment bank) it gets to borrow from the fed freely at essentially no cost. So, in essence the government is backing what one must admit is a risky investment in a private internet company.

I fundamentally agree with Johnson's assessment, but it seems to me the problem is much bigger than this Facebook investment. One really interesting question is whether an investment bank can reasonably also be a bank holding company. Allowing a firm like Goldman to become a bank creates all sorts of problems and I suspect this is just the tip of the iceberg. Banks should operate in a fundamentally risk averse manner. Investment banks can't. It seems to me that not only is the current structure of Goldman a problem, but the *concept* of Goldman is a problem.

We cannot allow any institution to be both secured by the government, and in the business of taking large risks. Of course defining "large risk" is problematic in that no one would have thought mortgages could, in any context, be defined as high risk. But while there may be gray areas where valuable collateral turns out to be not so valuable, an investment in Facebook ain't gray. And obviously much of Goldman's other investment banking related endeavors couldn't be defined as low risk either.

My point here is that I think Goldman needs to be clearly free to fail. That means we need to regulate them in such a way that we can guarantee that failure *is* an acceptable option. Regulators suck (see Bernie Maddoff) but in this case the not so free market sucks even more.

In light of the recent remote PHP exploit, I decided to update a couple servers I manage to ensure they weren't vulnerable. In each case, I had been using hand-compiled PHP builds, but decided that I'm simply too busy lately to be trying to maintain updates -- so I decided to install Zend Server. I've been using Zend Server CE on my laptop since before even any initial private betas, and have been pretty happy with it -- I only compile now when I need to test specific PHP versions.

One thing I've never been happy about, however, is that by default Zend Server exposes its administration GUI via both HTTP and HTTPS. Considering that the password gives you access to a lot of sensitive configuration, I want it to be encrypted.

Continue reading "Making Zend Server Available Via SSL Only"

NetTuts.com is featuring an in-depth thematic review of YUI Theater’s 2010 videos — a year in which we explored many of the most crucial themes in the world of frontend engineering.

Housing Minister Grant Shapps has said he wants to see more self-built homes.  Great, I want to see more sunshine, but just because I want to see it doesn’t mean it will happen.   Let’s examine how I will get more sunshine in the UK.  First, I will need to relocate the country many miles South, closer to the equator and/or invest heavily in weather changing chemicals I can fire into the air, or I could send up a couple of satellites with large mirrors to channel sunlight to the UK.

Mad ramblings I hear you cry, but no more far fetched than Mr Shapps dreams.

He may have been praised by some of his peers, including Labour, but it is all in my humble opinion just back slapping for expressing his opinion.   Yes he wants to tackle house prices, mortgage lending, first time buying and all the other political sabre rattling issues, but who doesn’t?!   It is unlike me to be negative of people with great ideas, but politicians fall into a different category and should only be judged on action.

The Community Right to Build is the flagship under which the government plans to bring about change and create what it calls ‘The Big Society’ (whatever that means).  Apparently the drive will  ‘… Allow communities to get together and take forward developments for new homes, shops and facilities in their area.  Allow a community organisation to go ahead with development without the need for an application for planning permission, if there is overwhelming community support for the development and minimum criteria are met…’   This is straight from the government flyer.

I can’t help but think of the TV series ‘the thick of it‘ whenever I hear of such schemes and read such dribble.

Indeed, there are mixed messages.  On the one hand, the government has advised planning authorities that we should not be allowed to sell off or build on on our own land (back garden building).  Furthermore, Mr Shapps is saying in an interview in the Observer that homeowners should no longer view property as an investment that will see them through retirement, but as a roof over their head (ifaonline).  Then on the other hand he is saying, let’s promote self build, break down the barriers of planning, kick start lending for such building and create community building projects.

The EAT quotes Ted Stevens, chairman of the NSBA, as saying: “We believe there are hundreds of thousands of people in the UK that are keen to build their own, very affordable homes, so anything the Government can do to help make this happen has to be applauded. By encouraging more self-build, people will get the homes they really want, very cost effectively. For example, it’s perfectly possible to build a three-bedroom home for around £150,000.”

Yes it is possible to build a home for this amount (even for less), but one has to acquire the land first and does anyone really believe that Mr Shapps will actually (a) force comprehensive change to planning legislation and (b) force banks to lend to self builds?  Forcing banks to lend to anyone at the moment would be a better start!

As Alex Morton in the Spectator says of Shapps and Co:  ‘…Promises of intent are one thing, accountability for decisions another…’  (discussing house price stabilisation).

Web and iPhone maps provider CloudMade can add a few more platforms to that list. Today it acquired German geospatial company One Step Ahead to incorporate Android, MeeGo and other mobile platforms to its suite of developer tools. CloudMade will also incorporate One Step Ahead’s offline/online map technology, which passes a few map updates at a time.

CloudMade, which has several CloudMade APIs, has found developers more likely to use device-specific mobile SDKs, according to Nick Black, CloudMade’s Head of Products. “CloudMade today is focused on the iPhone, but cross platform is becoming increasingly important for mobile developers,” Black said. That’s where One Step Ahead comes in, as the company already has an offline navigation app for Android. An early version is demoed in the video below.

The offline maps were originally built to get around European mobile internet roaming charges. When all the data lives in the device, another outcome is that you can zoom, pan and search without the lag normally seen while waiting for the result from a service. One Step Ahead has also created offline/online hybrid maps that will receive new data when a connection is available. The new data might be changes to the map since the last download or to cover a new area where the device does not have a complete map. Rather than downloading entirely new data every time, as many maps do, offline/online hybrids “trickle little bits of updates,” Black said.

It’s complex technology to implement the trickling of updates and rendering maps from data on the device. CloudMade is probably right to abstract the process from the developer. When it comes to mobile, that’s becoming a common way to distribute APIs, whether or not the underlying service is also made available.

One Step Ahead’s Stuttgart headquarters will become CloudMade’s second engineering office (the first is in Ukraine). The technologies will be merged in the next couple months, according to Black.

Sponsored by

Well, I clearly have a reputation for being a pedant, somehow. And so people keep asking for advice on VAT!

So if you are charging for services for January - what VAT rate do you charge?

The rules are in the VAT Act, and have not changed. They are the same any time the VAT rate changes.

It is suppliers choice, not customers. You can either charge based on the "actual" tax point or the effective tax point (e.g. invoice date). So one of these :-

1. Invoice date before 4th Jan, charge VAT on everything at 17.5%
2. Invoice date on or after 4th Jan, charge VAT on everything at 20.0%
3. Split VAT charges so services pre 4th are at 17.5% and those on/after 4th are at 20.0%

What you can't do :-

1. Invoice before 4th Jan, charging VAT at 20% for services before 4th.
2. Invoice on/after 4th Jan, charging VAT at 17.5% for services on/after 4th

Simples.

(well, almost, special case: if invoice to a connected party (e.g. director or director's wife) who cannot reclaim VAT, charged at 17.5% for any service on/after 4th Jan, then you have to invoice an extra 2.5% VAT. Finance Act (No.2) 2010.)

Around the corner we have the brand-spanking-new web server that will be running WebPagetest.org (among other random personal sites).  The Voip converter just happens to be there because that's where my phone line comes in and it's great for blocking telemarketers and keeping the phone from ringing when the kids are sleeping.


Finally, we have the heart of the data center tour, the network that pulls it all together. I've been completely spoiled by FIOS.  Seriously low latency high bandwidth connectivity right to the home.  The bulk of that wiring is really just my house and random devices (yes, all of the plugged in ports are live - everything has a network connection these days).  The BSD router is overkill these days.  It was there originally because the traffic shaping was done there but now that it has moved into the testers themselves I need to get around to replacing it with something a little less power hungry.


And there you have it.  The Meenan Data Center in all it's glory!

I again work with the system which needs high insertion rate for data which generally fits in memory. Last time I worked with similar system it used MyISAM and the system was built using multiple tables. Using multiple key caches was the good solution at that time and we could get over 200K of inserts/sec.

This time I worked with Innodb tables… it was a different system with different table structure, not to mention different hardware so It can’t be compared directly, still it is nice to see you can get the numbers as high with Innodb too.

I will spare you all experiments we went through and just share final numbers. On 8 core Opteron Box we were able to achieve 275K inserts/sec at which time we started to see load to get IO bound because of log writes and flushing dirty buffers. I’m confident you can get to 400K+ inserts/sec on faster hardware and disks (say better RAID or Flash) which is a very cool number. Of course, mind you this is in memory insertion in the simple table and table with long rows and bunch of indexes will see lower numbers.

So what’s the deal ? First MySQL 5.5 (frankly I did not try Percona Server 5.1 in this case) With MySQL 5.1 and Innodb Plugin we could see 40%+ CPU wasted on mutex spinlocks (per oprofile), which went down to about 15% in MySQL 5.5.8 with 8 concurrent threads. This both shows there is a substantial gains as well as room for more performance optimizations. Dmitri has good suggestions on tuning MySQL 5.5 and this is what I used for start. Using multiple buffer pools with innodb_buffer_pool_instances=8 was very important.

Second thing – Partitioning. Unfortunately MySQL 5.5 leaves the huge bottleneck for write workloads in place – there is per index rw lock, so only one thread can insert index entry at the time, which can be significant bottleneck. We got 2x+ better performance by hash partitioning table by one of the columns and I would expect gains can be higher with more cores. PARTITION BY HASH(col) PARTITIONS 8 is what we used. This looks like a good workaround but remember partitioning can impact performance of your select queries dramatically.

The inserts in this case of course are bulk inserts… using single value inserts you would get much lower numbers. In fact we used load data infile which is one of the ways to get a great performance (the competing way is to have prepared bulk insert statements).

We need to try new Percona Server 5.5 on our Cisco box to see if we can get to 500K inserts/sec – this can be a nice round number

---

Entry posted by Peter Zaitsev | No comment

Add to: | | | |

I often wonder what's behind the increased trend behind Hadoop and other NoSQL technologies. I realize if you're Yahoo that such technology makes sense. I don't get why everyone else wants to use it.

Reading Stephen O'Grady's self-review of his predictions for 2010 for the first time gave me some insights into how such people think:

Democratization of Big Data

Consider that RedMonk, a four person analyst shop, has the technical wherewithal to attack datasets ranging from gigabytes to terabytes in size. Unless you’re making institutional money, budgets historically have not permitted this. The tools of Big Data have never been more accessible than they are today.

read more

PlanetMySQL Voting: Vote UP / Vote DOWN