Alexandre Bourget’s energetic presentation on Pylons is at last online: Pylons, Web development done right.

Your browser doesn’t support HTML5. Please use the download link.
If you use Safari and want to use a libre format, install the Xiph QuickTime Component at http://www.xiph.org/quicktime

I’ve obviously been quiet here on my personal blog – as everyone who reads regularly knows I’m neck-deep in a pretty exciting startup call Nasuni as well as doing other projects, like the PSF Sponsored sprints thing. That combined with twitter means my time for other additional long-form content is minimal. So here’s a small roundup of interesting things:

Nasuni

Yup, still running Python and Django! We’re actually pretty proud to be a sponsor for DjangoCon 2010 coming up in September – I’ll be attending, so I hope to see all the familiar Django faces I know, and meet some new ones.

I’ve been blogging semi-regularly for the Nasuni blog itself – my posts are focused on product-things more than anything else. Here’s a small list of posts which I’ve done:

• The Road to Release – Feature Previews – this is actually my latest one, and the first in a series where I’ll be showing off some of the new features we’re adding in the latest release.
• Looking at OpenStack, a Rackspace and NASA initiative – For those of you who don’t know, Rackspace and NASA announced OpenStack – the awesome part? It’s all python – I had the swift component (which powers Rackspace’s cloudfiles system) of OpenStack running pretty quickly. I’d recommend snagging the code from launchpad and taking a look. Swift (the storage component) uses eventlet – and Nova (the compute part) uses Tornado and Twisted.
• Storage Switzerland Test Drives the Filer – This is a response to an article written about the product – I actually used it to preview some of the work going into the next release of the Filer.
• Thanks to Django – This piece goes into some detail about our use of Django, it’s one of our ways of saying thanks. I still need to rework it so we can send it over for the Django Success Stories page.
• Thanks to the Supporting Cast – This is an earlier thank you post – but to the other people who have helped out a ton, including Greg Newman, Lincoln Loop, and Revsys.
• The Donut Solution – This was a fun one, mainly to show that yes – we’re listening hard to customer feedback, and we’re improving/iterating quickly. Also, I get to show off UI improvements.
• Finally – The Nasuni Blog team – this is the rosetta stone for the authors of the blog, describing who we are. I didn’t write this piece, but it’s good reading to figure out who is who.

If you’re interested in Nasuni – or cloud storage in general – I’d encourage you to sign up for the RSS feed. We’re trying to keep the information useful outside of “just us” (despite my urge and predilection to churning out completely product-related posts) – and if you ever have feedback, drop us a line.

PSF Sponsored Sprints

The project continues on – we’ve funded two sprints so far, and have several more coming down the pike. We’re always in need of volunteers to help us do things like the manuals and site maintenance/content authoring. Here’s some highlights:

• The call for applications is open – The call for applications is open – and now I suspect we won’t be closing it. Originally, I thought we’d have to do things in waves of apply-approve. As time has progressed, I no longer think this is the case.
• Montreal Python Packaging sprint wrap up – the wrap up report for our first sprint!
• Europython core sprint report - another wrap up report for the core sprint we provided funds to.
• Just added the locations page – we now have people/companies offering up space for sprinters! Check it out!
• Finally - Sprints at PyOhio – PyOhio is going on this weekend, if you’re in the area you should really go check it out! Catherine has gone above and beyond with the entire “become a contributor” effort going on.

Please! If you’re thinking about holding a sprint - send us an application! Heck, even if we’re not sponsoring it, we’ll help promote you via the blog, and the sprint calendar we have up. A little fact? The sprints we’ve funded so far, and that are on deck for funding are all outside of the US, which is both awesome, and surprising!

PSF Board

Some of you probably know that I’m currently on the board of directors for the PSF – things progress well here, but I mainly wanted to call out the excellent blog Doug Hellmann has been authoring for PSF news. You should really be watching that because yes – we do do things, and hopefully over the next year, we’ll be doing more awesome things.

I’ve actually got a bigger post in the works for what I think the ultimate mission of the PSF is/should be as well as “how do you get money from us” as well. Must find the time!

Today we released LFS 0.5.0 beta 4

What's new?

• Bugfix pages: caching page_view
• Bugfix cart: display correct stock amount within growl message
• Bugfix product_inline: display property title within error message
• Bugfix order_received_mail.html: display the correct selected values of a configurable product
• Bugfix cart: calculation of maximum delivery date
• Bugfix redirect: save redirect url for variants
• Bugfix lfs.page.views: added missing import of Http404
• Bugfix: restrict adding to cart if the product is not deliverable. Issue #37
• Added french translations (Jacques Seite)
• Added get_properties method to OrderItem
• Added optional cached parameter to cart/utils/get_cart_price and cart/utils/get_cart_costs
• Removed javascript which dynamically sets the height of the slots.
• Changed properties management: display name instead of title within left portlet
• Improved lfs.portlet: caching

Further Information

You can find more information and help on following places:

• Official page
• Documentation on PyPI
• Demo
• Releases on PyPI
• Source code on bitbucket.org
• Google Group
• lfsproject on Twitter
• IRC

Hookbox is a new Python-based comet server with support for Websockets. It’s main features include:

• Support for named channels on which data is published/subscribed.
• Server-side Websocket and WSGI support via the eventlet concurrent networking library.
• Client-side Websocket support via js.io.
• Fall-back to a custom comet protocol when websocket support is not available in the browser.
• A RESTful api to interact with the channels.
• Easy integration with web frameworks like Quixote and Django, again via web hooks.
• A built-in admin interface to view the state of active channels and users.

It has been authored by some of the original people behind Orbited and represents a significant simplification in the process of writing websocket-enabled web apps primarily because it includes its own message broker.

Experiment: Graphic EQ

To better understand Hookbox, I decided to write a web-based graphic EQ which updates itself based on data produced on the server (strictly speaking, a graphic EQ is a control instrument, but my example presents it as a measurement instrument so perhaps calling it a web-based spectrum analyzer would be more appropriate). Checkout the video below to see it in action:

Hookbox Graphic EQ Demo from Salman Haq on Vimeo.

The top-half of the clip shows the equalizer bands dynamically updating in Firefox (v3.6.8) while the bottom-half shows the Firebug pane displaying the live network calls. If you look carefully, you will see that in this particular experiment all the network requests used the CSP protocol and not the websocket protocol. This is expected because websocket support is only available in version 4 or later of Firefox.

The video capture was a bit sluggish so it’s worth mentioning that it actually ran much smoother than it appears in the video. This is due to the fact that graphics acceleration is improperly configured on my quad-core box. (Side note: Each time I upgrade to the latest Ubuntu release, my video and printer settings need to be tweaked significantly to recover optimal performance. Since I upgraded to Lucid Lynx, I have not been able get the video performance I would like).

Trials

The experiments were repeated using three data publishing rates: 10 Hz, 100 Hz, and 1000 Hz. I should note that the producer.py process used time.sleep() which is not very precise on systems running a non-realtime kernel which typically lack high-precision timers. At the 10 and 100 Hz rates, the browser was able to keep up with the deluge of data and rendered the bands quite smoothly. At the 1KHz rate, all of my system’s cores reached maximum loading and Firefox rendered the bands rather slowly. The memory consumption did not jump between these trials and the network bandwidth increased slightly but never crossed 30 kbps in the steady-state.

These observations suggest that the experiments were CPU-limited rather than memory or bandwidth limited as the data publishing rate was increased. This is fitting with my earlier remark that graphics acceleration on the test system is mis-configured, which means that the display rendering was done by the CPUs rather than the onboard video card and hence the high cpu consumption.

I failed to repeat these experiments with Google Chrome due to this bug which I consequently filed on github. But given that V8 is a faster javascript engine, I would expect Chrome to perform better.

Code

The source code of the experiment is available in my fork of the hookbox project on github (I expect it will be pulled into Michael Carter’s main branch soon). It consists of fewer than 100 lines of Python, fewer than 30 lines of Javascript, and took about a couple of hours to get working satisfactorily. That’s not bad considering that it was an introductory play-date with Hookbox.

The equalizer widget consists of seven sliders from the jQuery-UI library, lined up vertically alongside each other. To update the sliders, a call back is attached to the hookbox subscription object’s “onPublish” event which looks like this:

                // see index.html for the full code.
                subscription.onPublish = function(frame) {
                  // adjust each slider from the data values in the frame's payload.
                  // the payload is expected to be an array of seven integers (one for
                  // each slider).
                  $("#eq > span").each(function (index) {
                        $(this).slider("value", frame.payload[index]);
                  });
                };

The data production and publishing loop is also very simple. Seven random integer samples are produced, and sent via a HTTP POST to the Hookbox server, which then publishes it to all subscribers of that channel:

   # see producer.py for the full code.
    while True:
        # generate seven random data points
        # and publish them via the HTTP/REST api at 10 Hz.
        # pop = range(0,100) (defined earlier)
        # 'values' is a dictionary with some other fields required by hookbox (also defined earlier).
        values["payload"] = random.sample(pop, 7)
        data = urllib.urlencode(values)
        req = urllib2.Request(url, data)
        # publish the samples!
        resp = urllib2.urlopen(req)
        time.sleep(0.1)

The common and most important thing between these two snippets is the “payload” object. In this example, it is an array of seven integers, ranging from 0 to 99 that is produced by the aptly named producer.py which also publishes it via the hookbox server and is received by the javascript running in the browser. The rest of the code in my example is quite self-explanatory and is a good way to learn Hookbox (and even Quixote).

Conclusion

While still in its infancy, Hookbox looks like a very promising project. From a developer’s perspective, it allows you to get something up and running very quickly. Its simplicity derives from the web hook api which also concerns me a bit. Using HTTP for inter-process communication, as in my example producer.py communicates with the hookbox server to publish data, doesn’t seem like the best choice for low-latency high-throughput applications. Perhaps Hookbox might introduce a web socket interface for inter-process communication so that the data can travel from the producer to the consumer using only one protocol? Overall, I’m quite impressed with Hookbox and really hope that it gains momentum. And lastly, don’t forget to read Michael Carter’s introduction to the project on cometdaily.com!