It’s doubtful that anybody reading this blog missed the news that Apple finally took the wraps off their much rumored tablet: the iPad. Trouble is, a bunch of folks seem to be upset about the features and specs, or something that made the buzz machine go meh. It’s just a bigger iPhone, complain the privileged tech pundits.

They apparently missed the recent Pew Internet Project report on internet usage by demographic. While it shows white users most frequently access the internet from home, black and hispanic users more frequently get online from mobile devices. Further, internet use by hispanics jumped dramatically in recent years, far exceeding the growth among whites.

The report further notes that while 83% of US adults have cell phones, only 60% use the internet from home. I’ve said it before: our notions of what a “computer” is have to change. The age of ubiquitous connectivity, Twitter, Facebook, and uncounted other tiny miracles has already changed the the reasons we use technology and shown us the difference between what it’s for and what it does.

The Pew stats show our computers as historical artifacts of a different age, built for a different purpose. The iPad is built for the ubiquitous social internet. The iPad is built for everybody who enjoys mobile internet access and the remaining 40% of users who don’t have any, though I’m quite certain that experienced internet users will eventually fall in love with the device too. Remember, the then leading tech news site Slashdot panned the original iPod in 2001: “No wireless. Less space than a Nomad. Lame.”

You might have to check Wikipedia to remember what the Nomad was today, though the manufacturer once enjoyed 65% market share. The market for MP3 players in 2001 was just under $2 billion, by 2006 it had tripled to almost $6 billion. iPod sales continued to grow, much to the annoyance  of iPod haters, until Apple released the iPhone and started cannibalizing their traditional iPod sales. Convergence had finally arrived.

Apple’s plan with the iPad is to dramatically expand the market for internet connected devices. Do you really want to bet against them?

WordPress has some simple built-in support for posting by email, but that didn’t stop a couple people  from developing plugins that might do better. Postie and PostMaster both claim to support attached photos (though neither appears to use WP’s built-in media management). But if your goal is to post photos, you might consider posting through Flickr.

As more and more organizations become aware of the need to track their online reputation, more people in those organizations are following Google alerts for their organization’s name. That creates a perfect opportunity for scammers to play on that organizational vanity to infect computers used by officers of the organization with malware that can reveal the inner workings of that organization.

I’m not exactly sure what clicking the button above does. The sketchy domain name is concerning enough to keep me from clicking, honestly. The page includes an embedded Flash file (the red button), and I’d suspect that somebody’s hoping that people are running un-patched versions of the Flash plugin. For those running vulnerable versions of Flash, simply visiting the page opens the door to malware infection. The user is at risk even if s/he doesn’t click the button.

The effect of this type of attack is not insignificant. The attacks on Google that led to the company’s dustup China this week are said to have depended on social engineering and affected as many as 30 other companies. An October 2009 Northrop Grumman report on Chinese hacking activities directed at the US suggests as much as 20 terabytes of sensitive data has been exfiltrated using similar attacks.

A post on thomas fitzgerald.net serves to remind us that Apple released their first netbook in 1997: the Apple eMate 300:

…next time you see people ranting about an Apple netbook, remember that Apple had something similar long before anyone even uttered the phrase “netbook.”

The device ran Netwon OS 2 with a 20-30 hour battery life (yes, 20-30 hours). I’ve written more than a few posts eulogizing the eMate’s tablet-shaped sibling: Newton Message Pad 2000.

I’d long been a user of BareBones’ BBEdit, a product that’s served me well for a number of years. But upgrading from version 8.5 to 9 is a paid deal, and after spending 15 days with the demo of BBEdit 9, I decided I wanted to look around a little bit. My friend Matt switched from BBEdit to Panic’s Coda some time ago, and I liked the demo of that well enough that I bought a license.

Now, after using Coda for about a month, I’ve found that I miss a number of BBEdit’s behaviors and can point to a few areas where Coda can use improvement.

Selection ranges

If you don’t know that double clicking a word is the fastest way to select the entire word in most GUIs, you’re missing out. However, the rules that define a “word” (or the word boundaries) vary considerably. BBedit’s rules are clear and consistent, but Coda does weird things. Specifically, double-clicking a variable like $var does not select the $ in BBEdit, but Coda’s selection behavior changes. If you simply double-click the $var, Coda will select the $. But, if you double-click, then move the mouse to extend the selection range, Coda drops the $. Why? This has led to a number of compilation errors because of missing $s preceding variable names. Argh.

(Worth mentioning: text selection in Windows apps is the worst of all, that’s the biggest reason why I can’t use Windows.)

Find, grep, regex

The marketing materials for both BBedit and Coda praise their find and grep features. I was pleasantly surprised at first to see Coda’s rich and easily accessible options for those features, but then I began to miss a number of BBEdit’s features and options.

The simplest different is that BBEdit gives much more space to create both search and replace patterns. Both support regular expressions, but BBEdit’s support for non-regex searches is stronger. If I continue to use Coda, I’ll need to develop the practice of always writing my search and replace expressions as regex, which wouldn’t be so bad if I could understand Coda’s regex engine. Coda offers a number of options to control the regex engine, but all I want is to duplicate PHP’s preg_match() behavior.

FTP behavior

While my first two complaints related to deficiencies in Coda as relates to BBEdit, this is probably the feature in Coda that made me switch. The integrated FTP works well and answers a number of complaints I had with BBEdit. Creating a new remote file is a surprisingly difficult task in BBEdit, but works simply and conveniently in Coda. That said, I have a few frustrations.

A few examples:

• Navigating the remote filesystem isn’t as fast as it should be; hitting the enter key on a selected file or directory renames it rather than opens it (big complaint). This behavior mimics the Finder, but contradicts a number of conventions of the File > Open dialog.
• Copying a file from one directory to another isn’t as easy and simple as it should be. I wouldn’t complain about this if the above behavior were less like the Finder’s in other respects (that is, I’d rather solve the first issue and drop this one).
• I’ve grown rather accustomed to Mac OS X’s Quick Look feature. Why can’t I quick look the images and other non-text files in the FTP viewer? (Another small complaint.)

Update: Panic has a blog (and the design is awesome), but I don’t see any obvious place to leave product feedback.

I spied the Wintec FileMate 48GB Ultra ExpressCard and began to wonder how it works as a boot drive for Mac OS X in a late 2008 MacBook Pro (the model just before Apple replaced the ExpressCard slot with an SD slot). But I didn’t have to wonder too much, as a post to this MacObserver forum thread offers enough details to make a geek salivate:

The computer now boots primarily from the SSD Card and will start up the computer in less than 1/2 the time of the internal HD [...] I have all the applications and system files on the SSD Card, the user files/record on the internal HD. Programs launch about 4 to 5 times faster.

The manufacturer claims 115MB/s reads and 65MB/s and writes, which is better than the max performance (according to Tom’s Hardware) of the 320GB 5400 RPM drive it shipped with, but only similar to the max performance of the top of the line 7200 RPM drives. The stated read performance of the SSD, however, is only slightly better than the average performance those 7200 RPM drives, and the write performance is middling. For comparison, Intel claims their consumer-class X25-M can do 250MB/s reads and 100MB/s writes. (However, none of these numbers show the effect of seek times on actual performance, which may account for the gains reported in the forum posting.)

So, is it worth it?

David Weinberger kicked off the latest installment in the ongoing debate about the future of electronic books versus paper books in his Will books survive? A scorecard… post.

He’s got some good points, but like many of the smart folks I admire, he approaches this question assuming that books, in any form, are important. Ursula K. Le Guin’s excellent essay on “the alleged decline of reading” is especially informative on this point: books don’t matter to most Americans, and they haven’t for some time.

And among those who do read, the book industry’s bread and butter is in romance novels that appeal “largely older, less affluent female buyer.” The continued commercial viability of country music radio stations while other formats suffer declines blamed on iPods might suggest that those readers are technology averse (yes, I’m assuming country listeners and romance readers are a similar demographic), but just as that audience learned how to use VHS, then DVD, they’ll likely learn to appreciate other book formats as well.

A few of David’s readers have commented that books will likely become the new vinyl. They’re probably right, but that doesn’t mean ebooks will replace them in the mainstream. As noted in my comment, I’m not sure paper-bound books will survive the cultural and economic shifts that face them long enough to be effectively translated into some electronic form.

I sometimes wonder if we’ve already replaced the book and it’s called “blog.”

Ethan Zuckerman’s recent post, What if they stop clicking? points out the difficulty of building a business on ad revenue. He points to statistics that show fewer readers are clicking banner and arguments from the web advertising industry about how un-clicked ads still build brand awareness.

It’s not really central to Zuckerman’s point, but I didn’t sense that he was aware that Google has picked up the same argument. I commented on the post that Google has started reporting the numbers of people who are presented (but don’t click) ads, then later visit the advertisers that are paying for, um, clicks.

On the one hand this is probably supposed to make advertisers feel better about spending the money, on the other hand, I can imagine Google finding a way to charge for those visits too.

Zuckerman seems to genuinely doubt that advertising can support anything other than search engines, and he may be right. The internet itself is the enemy of ad supported business models. Old newspapers were discarded, but old web pages get new ads for every visitor. Given the number of places an ad can run (and the thousands of new pages Demand Studios pumping out each day), there’s no way to create the scarcity that made print ads so valuable. Meanwhile, the abundance of content on the web forces us to develop the very skills we use to ignore ads and other irrelevant material.

Aside: Google, please consider de-ranking eHow and other Demand Studios properties. Their content was frustrating before I understood how evil their model was, now it’s worse.

Authentication Hacks

My first talk was on User Authentication with MU in Existing Ecosystems, all about integrating WP with LDAP/AD/CAS and other directory authentication schemes, as well as the hacks I did to make that integration bi-directional and deliver new user features. My slides are online (.MOV / .PDF), and you can read earlier blog post summing up the project.

Plugins Mentioned

• wpCAS (long description)
• Alternate Contact Info
• WordPress Ticket Framework
• wpSMS (long description)

Scriblio

I was most excited, however, to talk about Scriblio, a plugin that turns WordPress into a library catalog with faceted searching and browsing. Those slides are online as well (.MOV / .PDF). The core plugin is in the repository, but I’d recommend people join the mail list if they’re thinking of diving in to it.

Scriblio Sites I Demoed

• Collingswood Public Library
• Colby-Sawyer College Archives
• Beyond Brown Paper photo archive

Matt demanded accent-aware spell checking for the WordPress spell checking plugin his company acquired earlier this year. And just a little more than a month later, After the Deadline delivered. Now Beyoncé, café, coöperate, and even my resumé look prettier.

Separately, Wordnik offers a new take on online dictionaries, and they just launched an API.

Backblaze is a cloud backup service that needs cheap storage. Lots of it. They say a petabyte worth of raw drives runs under $100,000, but buying that much storage in products from major vendors easily costs over $1,000,000. So they built their own.

The result is a 4U rack-mounted Linux-based server that contains 67 terabytes at a material cost of $7,867, the bulk of which goes to purchase the drives themselves.

And best of all, they open sourced their hardware:

I’ve been a fan of Drobo since I got mine over a year ago. The little(-ish, and sweet looking, for stack of disks) device packs as many as four drives and automatically manages them to ensure the reliability of your data and easy expandability of the storage. However, Thomas Tomchak just pointed out one major flaw: if you overflow your Drobo with data, the entire device may give up and you’ll lose everything.

How do you overflow a Drobo? Most users only have a few terrabytes of storage in their Drobo, but configure it to tell the computer its attached to that it can store eight or 16 TB of data. Doing that allows easy expansion when more or larger drives are added — the attached computer doesn’t need to reformat anything, it can simply save more stuff to the device — but it also opens the door to the Drobo overflow.

From Tomchak’s post:

While on my tech support call I asked the engineer how frequently he received calls about this particular problem. After a big sigh he admitted that it was nearly every day.

One commenter on the article suggested the Drobo could “just simulate that the uninstalled part is already full of simulated read-only data,” a suggestion that makes sense, but may require the Drobo to know more about the filesystem on it than it otherwise would.

I’ve been at 90% capacity on my Drobo for a while, I think it’s time I popped another disk in there.

(CC licensed photo by Pixelthing.)

A few years ago I found an article pointing out how spammers had figured out how to abuse some code I wrote back in 2001 or so. I’d put it on the list to fix and even started a blog post so that I could take my lumps publicly.

Now I’ve rediscovered that draft post…and that I never fixed the bad code it had fingered. Worse, I’m no longer in a position to change the code.

Along similar lines, I’ve been told that a database driven DHCP config file generator that I wrote back in the late 1990s is still in use, and still suffers bugs due to my failure to sanitize MAC addresses that, being entered by humans, sometimes have errors.

I’ve written bad code since then and will write more bad code still, but as my participation in open source projects has increased, I’ve enjoyed the benefit of community examples and criticism. My work now is better for it.

Most of my work is available publicly, but some development is hosted on a private SVN that’s hidden behind a firewall. Unfortunately, my primary development server is on the wrong side of that particular firewall, so I use the following command to bridge the gap:

ssh -R 1980:svn_host:80 username@dev_server.com

That creates a reverse tunnel through my laptop to the SVN server and allows me to checkout code using the following:

http://localhost:1980/path/to/trunk

I’m posting that because I lost my terminal command history and had to think for a moment about how to do this again.

Years ago I used to tunnel my outgoing email to an un-authenticated SMTP server that only accepted outgoing messages from hosts on the local network. That was fairly common back in 2000 or so, but obviously made life (or communication) difficult for people at home or on the road. The easy solution was to SSH to a machine on mail server’s local network and forward emails through it.

ssh -L 1925:email_host:25 username@ssh_host

Doing that, I was able to configure my mail client to send outgoing emails using a server configuration like the following:

SMTP host: localhost
SMTP port: 1925

Search Engine Land.com:

Yelp…is…essentially a poster-child for semantic markup. This spring, Google’s introduction of rich snippets has allowed Yelp’s listings in the SERPs to stand out more, attracting consumers to click more due to the “bling” decorating the listings in the form of the star ratings.

There are now some very good reasons why sites with ratings and reviews should be adopting microformats, and it’s not that hard to do! For a more detailed explanation, read my recap on the subject, Why Use Microformats?

In March of this year Apple applied for a patent on technology that enables or disables features of a phone via a config file. The tech is already in use: it’s the carrier profiles we’ve been downloading recently. On the one hand this is just an extension of the parental controls that Apple has included in Mac OS X since the early days, but it also implies some rather anti-consumer thinking at the company.

One examplar claim in the patent is that the config file can include a “blacklist of device resources to be restricted from access.”

AT&T used this this technology to block MMS until recently, and uses it now to block tethering, but the description given in the patent application goes much further:

For example, a carrier may wish to provide an enhanced service which utilizes the global positioning system (GPS) functionality in a mobile device. Carrier may wish to charge a premium for this service, so it may configure carrier provisioning profile to disallow third party applications from accessing the GPS functionality in device, and instead only allow applications digitally signed by carrier (or another entity affiliated with carrier) to access the GPS services in device.

Readers may remember the Trusted Computing video by Lutz Vogel and Benjamin Stephan that spotlighted the growing interest within the computing industry to impose new and artificial restrictions on the way we use the hardware and software we use daily.

A quick Google search of klaomta.com reveals more than a few people wondering why it’s iframed on their websites. The answer is that the site has been compromised.

Unfortunately for the fellow who asked me the question at WordCamp, solving the problem can be a bit of a chore. Keeping your WordPress installation up to date is important, as there are some known security flaws in older versions, but most of the attacks that crackers use are targeted elsewhere. Your passwords, all your server apps, the PHP config, your hosting control panel, and other users all must go under the microscope when trying to find security holes.

Plugin Development

Will Norris‘ talk at WordCamp PDX introduces WordPress coding standards, common functions, and constants to would be plugin developers (and smacks those who’ve already done it wrong). Also notable: functions, classes, variables, and constants in the WordPress trunk.

Custom Installations

Just as WordPress has a number of hooks and filters that plugins can use to modify and extend behavior, it also has a cool way to customize the installation process.

Extending The WYSIWYG Editor

TinyMCE, the WYSIWYG editor in WordPress has a rich API to allow adding buttons and stuff, but the docs are hard to get into. We can get a jump on that by looking at how it’s implemented in other WP plugins. This code creates the buttons, while the function that responds to the button click and does the work is defined within the plugin. The TinyMCE plugins in core are also informative.

The one way relationship between our user accounts and the MIS database also makes it difficult to engage with new users online. If you can’t get an account until you become a student, how do you allow potential students to apply online if all your systems are integrated with single sign-on? And if you can’t authenticate the online identity of your users, how do you set initial passwords into your system? Or allow them to reset a forgotten password online?

Internet companies never struggled with this issue, as their customers could only approach them online, but most universities built systems around paper applications and have fond (and relatively recent) memories of offering their students their first internet experience. It’s still not unusual for universities to offer their students their campus computing account with a default password based on supposedly secret data shared between the user and the school. But your SSN, birth date, and mother’s name are no longer secret. A proposed change in FERPA policy (see the the top of page 15586 in the NPRM) would have barred the use of “a common form user name (e.g., last name and first name initial) with date of birth or SSN, or a portion of the SSN, as an initial password to be changed upon first use of the system” in systems that store academic data. The final rule excluded that provision, much to the relief of those schools with more lobbying clout than brains.

Platform Choices

Rather than wait to see how the ruling played out last year, we went to work trying to improve security while easing access to our systems (no, that is not self-contradictory). Our challenges were thus:

• Fix initial password assignment
• Fix password resets
• Allow users with a loose or undefined relationship to the institution to create limited accounts for the purpose of interacting with the institution or its members

We considered a number of paths to a solution, including hacking of our university portal (which hosts the CAS single sign-on in our environment), expansion of a limited home-built solution, and a review of commercial and open source products and frameworks. We simplified the problem by confirming that the FERPA rule did not require us to authenticate the “real life” identity of a person; rather, we had only to validate the online identity of a person (saving us from needing to do things like send confirmation PINs by postal mail to a person’s home address).

In the end, we chose WordPress MU. Significant factors were our experience with the software (all the MIS developers use it personally), the extensibility of it as an application platform, the development focus on user experience (especially in recent versions), and our interest in using it as a framework for other user-facing services (especially BuddyPress).

Our Needs vs. WordPress

• The system must serve as the front end to our single sign-on environment, using our AD and LDAP password stores to authenticate users who have accounts in those systems.
• External email addresses, once verified with some challenge/response, can be used to reset a password.
• Users who are presently affiliated with the school have a school-provided email address, but no external address with which to reset their lost passwords.
• Users who are not presently affiliated with the school have no school-provided email address, and must verify their external email address before their account is activated. They can then set their own password once they verify their email address.
• The ability to send password reset codes via SMS would be nice (especially considering the number of long-time employees of the university who do not have personal email accounts), though that also requires the verification of the user’s cell phone number.

After reviewing what we wanted to do, we surveyed WordPress’ code to develop an implementation plan. And, because a number of aspects of our application process were changing, we decided to focus on allowing current users to self-reset their password and postpone development of account self-creation features for new users. Still, a few issues quickly emerged:

• WordPress requires a username be assigned to each user, rather than relying on email address (this is likely to change in WP 2.9) Creating a new username for our users is unacceptable, but adding a large number of new users to our existing username space will quickly deplete the “good” usernames. And changing a user’s username as their affiliation with the institution is unacceptable.
• The core user authentication function can be replaced with our own function. (And in 2.8 it became filterable)
• WordPress MU will validate email addresses, but the system isn’t built to be extensible.
• WordPress only stores one email address per user, but the user meta system can be used to store a second one. Unfortunately (and in a manner inconsistent with post meta), only one value per meta key per user is allowed, making it difficult to allow users to have an arbitrary number of email addresses associated with their account.
• The function that identifies a user by a given email address can be replaced with a function that also checks the secondary address.
• WordPress user profiles have no phone field, but the user meta system can be used to store one. A function to identify a user by a given phone number must also be created.
• Unlike some settings pages, the fields on the user profile editor cannot be changed simply by modifying the $wp_settings_fields array.
• Upon doing a password reset, the user is sent a temporary password, rather than being allowed to set a new password. This contradicts University policy about how passwords are used and communicated and could train users that sending passwords by mail is acceptable.
• The various functions in wp-login.php cannot be replaced, and in WP 2.7 the code had no way to add or replace various login actions (WP 2.8 changed that).
• WPMU-specific functions don’t always follow WP coding standards or models.

(Note that we began our work and deployed the system under WPMU 2.7. WPMU 2.8 included a few changes that made the process easier. I’m proud to say that some of those changes were a result of code we offered back to WP during our development.)

What We Did

• We decided that email addresses (both PSU addresses and external addresses), as well as PSU usernames would be acceptable identifiers for an account, and that a person should be able to log in to our web services using any of those identifiers. So…
• We replaced wp_autenticate() with our own function that accepts either email address or university username, checks to see if the user exists locally, checks to see if they exist in AD or LDAP, confirms their password, provisions their WordPress account (for university users who’ve not logged in via this method yet), establishes a session with our university portal and redirects them there (unless $redirect is set to something more specific that the dashboard).
• We decided to replace WordPress’ usernames with a random string matching a pattern we established. This became the WPID. Doing this required us to hide references to username (easy if you set a preferred display name)
• To store phone numbers and secondary email addresses, and allow users to edit those within their profile, I created the Alternate Contact Info plugin (browse source). This requires more use of output buffering than I’d like, but it gets the job done.
• To confirm email addresses and phone numbers via a challenge/response message (and support other interactions), I created the WordPress Ticket Framework plugin (my introduction, browse source).
• To send messages via SMS, we used my wpSMS plugin (in the plugin directory, browse source).
• Matther Batchelder re-skinned the login screen via a plugin that inserts our custom CSS.
• After determining that our university portal could not be made to authenticate via CAS, I gave up work on my wpCAS Server plugin and developed another method to initiate the portal session (which then establishes a CAS session using the portal’s CAS server).
• We replaced most of the functionality of the wp-login.php page (by hacking core at first, then taking advantage of the action hook in 2.8). In doing so we were able to change the password reset behavior to allow users to immediately change their password after entering their reset code (which was sent to their email address or phone via SMS).

Over time we extended the system to host multiple domains and replace our CMS. Soon we’ll consolidate our  public blogging instance into it, and we’re building an invite system that we can use to invite people to join our community.

What It Looks Like

The re-skinned WordPress login

Entering an email address or username to get a password reset code

SMS text with password reset code

Enter the password reset code from the SMS text message here, or follow the link from the email

Extended contact information in the WordPress profile

And that’s how we replaced our authentication system with WordPress, gained self-service password resets, and built the foundation to invite new users into our system.

Moving data by homing pigeon requires some planning, and pigeons

It was a tech story so apparently humorous that the popular media felt compelled to cover it: carrier pigeons delivered 4GBs of data faster than an ADSL line. The BBC story’s subtitle read “broadband promised to unite the world with super-fast data delivery – but in South Africa it seems the web is still no faster than a humble pigeon,” and that’s how most stories played it. Unfortunately, they all got it wrong.

The race was run by The Unlimited Group, but the clearest telling of it comes from Wikipedia:

Inspired by RFC 2549, on 9 September 2009 the marketing team of The Unlimited, a regional company in South Africa, decided to host a tongue-in-cheek “Pigeon Race” between their pet pigeon “Winston” and local telecom company Telkom SA. The race is to send 4 gigabytes of data from Howick to Hillcrest, approximately 60 km apart. The pigeon carrying a microSD card (an avian variant of a sneakernet), versus a Telkom ADSL line. Winston beat the data transfer over Telkom’s ADSL line, with a total time of two hours, six minutes and 57 seconds from uploading data on the microSD card to completion of download from card. At the time of Winston’s victory, the 4GB ADSL transfer was just under 4% complete.

The critical numbers here are 4GB vs. 164MB (4% of 4GB) in 2 hours. Also, because the challenge was peer to peer transfer, rather than a simple download, the bandwidth is limited by both the upload and download speed of the technology. ADSL, like most consumer broadband technologies, offers slower upload speeds than download speeds.

Speedtest.net's summary of worldwide test results

My home ADSL line is claimed to be 3Mbps/768Kbps, but manages a little less than that in practice according to Speedtest.net (and their sweet iPhone app). That’s a little below average for the US (be sure to view upload speeds), but not dramatically so. And even the fastest region, Asia, can only boast 1.72 Mbps.

In the race between pigeon and internet, the 164MB network transfer over 2 hours is about 187 Kb/s. Significantly slower than my home connection is rated for. But even the fastest consumer upload speeds would have trouble beating the pigeon’s transfer speed. It would take sustained uploads of about 5Gb/s to transfer 4GB of data in the two hours of the pigeon stunt, a rate much faster than possible with the company’s 1Mb/s ADSL connection. But it isn’t just the South African internet that’s slow: The best performing “ISP” in New Hampshire, Dartmouth College, gets a 10 Mb/s rating in Speedtest.net, though very few others can manage over 4Mb/s (Comcast advertises upload speeds up to 10Mb/s, though that’s with “PowerBoost”; sustained speeds are lower, the company reportedly throttles FTP and other uploads, and Speedtest.net rates their business-class uploads at 3.47 Mb/s). If the story makes South African networks laughable, it makes the whole internet a comedy act.

microSD card sizes compared to other SD types

And the pigeon could have carried even more data and made a bigger win over broadband. microSD cards such as the one carried by the pigeon weigh less than half a gram and are now available in capacities up to 16GB (with a theoretical capacity of about 128GB). Assuming you can attached two such cards (one to each leg), you’d get an upload rate of about 40Mb/s in the South African test. Perhaps you could even attach four such cards (totaling under 2 grams of cargo and yielding 80Mb/s) or send more pigeons.

Consumer download speeds are typically much faster, and popular web services go to great lengths to ensure they fill those pipes to speedily deliver rich media. Apple was an early investor in Akamai, one of the first content delivery networks that speed downloads by distributing servers around the world so that your iTunes music and movie downloads are delivered from the closest servers and through the fastest pipes to your location. Netflix’s on demand service leverages the broad availability of these services to deliver TV shows and movies faster than the US Postal Service. But comparing internet speeds to postal service delivery of DVDs (a 14 year old standard) gives no real suggestion of attainable postal service bandwidth (especially when the network-delivered content is far more compressed than on DVD).

You could probably stuff about 30 MicroSD cards (which have almost five times the storage density of regular SD cards — 32GB in 1612.8mm³ vs 16GB in 165mm³) in a First Class letter ($.44 for 6.125 inch x 11.5 inch and 1 ounce) that gets delivered almost anywhere in the US in three days, yielding about 480GB in 72 hours or 15 Mb/s. A pair of 512GB SSDs, on the other hand, can be delivered over night, yielding 1TB in 24 hours or 97 Mb/s. Given that DVDs are 1.2mm thick and 120 mm in diameter, Netflix could easily fit as many as 80 microSD cards — 1280 GB (41 Mb/s over three days) — in their current delivery envelope, much more capacity than even the 50 GB (1.6 MB/s over three days) capacity of the largest Blue-ray discs. It’s for that reason that Amazon offers bulk import/export services of physical media for their web services.

Point: storage capacity has increased dramatically over the past few years, while internet speeds have remained relatively stable. The pigeon gimmick didn’t show us a laughably slow internet connection, it showed us how unimaginably vast our storage has grown (and yet we still find ways to fill it).

Photos: Winston, Speedtest.net, Wikipedia. History worth remembering: avian photographers (via Modern Mechanix).

Ben Fisherman’s JSNES runs entirely in the browser using nothing more intrusive than JavaScript. It apparently manages real-time performance within Chrome, but it works (if not playably) on an iPhone.

I wish the screen was resizable and that it supported iPhone compatible controls, but both of those assume that browser performance will improve enough to make it playable. Interestingly, though not surprisingly, the Safari JS engine is limited to consuming a single CPU (which it quickly does while playing JSNES).

Will Norris has discovered that iTunes 9’s interactions with the Store are more web-happy. I’ve been asking where the iTunes Store API was for some time, now I think I’ve got what I need to build one.

• site.org/blogname1
• site.org/departments/blogname2
• site.org/departments/blogname3
• site.org/services/blogname3

The association between blog IDs and sub-directory paths is determined in wpmu-settings.php, but the code there knows nothing about nested paths. So a person planning to use WordPress MU as a CMS must either flatten his/her information architecture, or do some hacking.

Challenge: hacking WordPress MU to support arbitrary directory paths for each blog

As with my multi-domain hack, the following assumes that you’re using the vhost=no setting, that you have access to and know how to manipulate your MySQL, that you have control over your DNS and know how to use it, and that you know how to configure Apache or similar. You’d also be smart to turn off any object caching you may have running, at least until we’re done doing direct database manipulation. The following also assumes that your wp-config.php sets the DOMAIN_CURRENT_SITE and PATH_CURRENT_SITE constants — if you’ve done a fresh install recently, it probably does, or you can check my domain mapping hack.

Hack The Path Mapping

Right at the top of wpmu-settings.php you can see how it strips all but the base of the URL path, but rather than mod that file, we can take advantage of an obscure MU hack: sunrise.php, which gets executed after some important WordPress components like the database class get loaded and before wpmu-settings.php.

To use sunrise.php, create a PHP file at /wp-content/sunrise.php and set define('SUNRISE', TRUE); in your wp-config.php.

Here’s the sunrise.php code I’m using:

if( defined( 'DOMAIN_CURRENT_SITE' ) && defined( 'PATH_CURRENT_SITE' ) ) {
	$current_site->id = (defined( 'SITE_ID_CURRENT_SITE' ) ? constant('SITE_ID_CURRENT_SITE') : 1);
	$current_site->domain = $domain = DOMAIN_CURRENT_SITE;
	$current_site->path  = $path = PATH_CURRENT_SITE;
	if( defined( 'BLOGID_CURRENT_SITE' ) )
		$current_site->blog_id = BLOGID_CURRENT_SITE;

	$url = parse_url( $_SERVER['REQUEST_URI'], PHP_URL_PATH );

	$patharray = (array) explode( '/', trim( $url, '/' ));
	$blogsearch = '';
	if( count( $patharray )){
		foreach( $patharray as $pathpart ){
			$pathsearch .= '/'. $pathpart;
			$blogsearch .= $wpdb->prepare(" OR (domain = %s AND path = %s) ", $domain, $pathsearch .'/' );
		}
	}

	$current_blog = $wpdb->get_row( $wpdb->prepare("SELECT *, LENGTH( path ) as pathlen FROM $wpdb->blogs WHERE domain = %s AND path = '/'", $domain, $path) . $blogsearch .'ORDER BY pathlen DESC LIMIT 1');

	$blog_id = $current_blog->blog_id;
	$public  = $current_blog->public;
	$site_id = $current_blog->site_id;
	$current_site = sl_get_current_site_name( $current_site );
}

function sl_get_current_site_name( $current_site ) {
	global $wpdb;
	$current_site->site_name = wp_cache_get( $current_site->id . ':current_site_name', "site-options" );
	if ( !$current_site->site_name ) {
		$current_site->site_name = $wpdb->get_var( $wpdb->prepare( "SELECT meta_value FROM $wpdb->sitemeta WHERE site_id = %d AND meta_key = 'site_name'", $current_site->id ) );
		if( $current_site->site_name == null )
			$current_site->site_name = ucfirst( $current_site->domain );
		wp_cache_set( $current_site->id . ':current_site_name', $current_site->site_name, 'site-options');
	}
	return $current_site;
}

The first few lines of the code do pretty much the same as the start of the wpmu_current_site() function in wpmu-settings.php, but starting with line 8 it takes a big departure.

That’s where it splits the requested URL path like /path/to/blog/and/stuff/ into pieces and constructs an SQL query against the wp_blogs table to identify the correct blog to serve the request. The following example shows how:

SELECT *, LENGTH( path ) as pathlen
	 FROM wp_blogs
	 WHERE domain = 'domain.org' AND path = '/'"
	  	 OR (domain = 'domain.org' AND path = '/path/')
	 	 OR (domain = 'domain.org' AND path = '/path/to/')
	 	 OR (domain = 'domain.org' AND path = '/path/to/blog/')
	 	 OR (domain = 'domain.org' AND path = '/path/to/blog/and/')
	 	 OR (domain = 'domain.org' AND path = '/path/to/blog/and/stuff/')
	 ORDER BY pathlen DESC
	 LIMIT 1

Optimization note

Setting a maximum depth (and array_slice( $patharray, 0, $maxdepth )) would allow the query to be cached up to that depth. Otherwise, the query must be executed for every page load. The $maxdepth could either be set arbitrarily, or could be determined automatically based on the maximum path length of registered blogs.

Setting Up New Blogs

Once you’ve hacked the path mapping (and tested that it didn’t break your current site), you can add a new blog at a nested path.

Create a new blog in the MU blog admin.

Unfortunately, MU strips the slashes from the URL path you just tried to set.

The new blog you just tried to create, but with a very different path.

Fortunately, you can set the path correctly in the MU blog editor, and it won’t break the path when you save there.

Set the blog path in the MU blog editor, MU won't break it when you save it this time.

Once you create the new blog, try to load it in your browser. You’ll quickly notice the stylesheet is missing, though the blog works and functions properly.

Hack The .htaccess

WPMU uses the following .htaccess rewrite rule to properly direct requests for files on the real filesystem:

RewriteRule  ^([_0-9a-zA-Z-]+/)?(wp-.*) $2 [L]

Obviously, that rule won’t work for deep paths, so I’ve replaced it with this rule:

RewriteRule  ^(.+)?/(wp-.*) /$2 [L]

And with that, you should be done.

Click here to view the embedded video.

With AT&T facing lawsuits for not delivering MMS features at the iPhone 3GS launch, they kind of had to do something. I’m not sure if I’d be satisfied by this video if I were among the plaintiffs, but I think it does a good enough job. The stat about 300% annual increases in mobile data use is pretty powerful. I’d heard it a dozen times before*, but because I wasn’t in Austin for SXSW iPhone meltdown, I don’t have quite the same appreciation as some do. AT&T added capacity then, and they seem to have been scrambling elsewhere too.

iPhone users are said to be six times as likely as anybody else to watch video on their phones, and if WiFi aggregator JiWire’s report says anything about cell data, the iPhone has certainly changed the game. JiWire’s Mobile Audience Insights Report shows that over 97% of the devices on their network are either iPhones (about 56% of the total) or iPod Touches! And all the way back in 2007 in Britain, iPhone users were 33 times as likely as other phone users to send or receive more than 25MB a month.

It will be interesting to see what happens to other carriers as they get devices that encourage use as the iPhone has.

*Actually, I hadn’t heard the 300% stat specifically, just inspecific reports of increased usage.

Click here to view the embedded video.

Okay, I don’t want to watch all the movies depicted in this 100 year overview of film special effects, but I did just add a few to my Netflix queue.

The full list, according to the description in YouTube:

• 1900 – The Enchanted Drawing
• 1903 – The Great Train Robbery
• 1923 – The Ten Commandments (Silent)
• 1927 – Sunrise
• 1933 – King Kong
• 1939 – The Wizard of Oz
• 1940 – The Thief of Bagdad
• 1954 – 20,000 Leagues Under the Sea
• 1956 – Forbidden Planet
• 1963 – Jason and the Argonauts
• 1964 – Mary Poppins
• 1977 – Star Wars
• 1982 – Tron
• 1985 – Back to the Future
• 1988 – Who Framed Roger Rabbit
• 1989 – The Abyss
• 1991 – Terminator 2: Judgement Day
• 1992 – The Young Indiana Jones Chronicles
• 1993 – Jurassic Park
• 2004 – Spider-Man 2
• 2005 – King Kong
• 2006 – Pirates of the Caribbean: Dead Man’s Chest
• 2007 – Pirates of the Caribbean: At World’s End
• 2007 – The Golden Compass
• 2008 – The Spiderwick Chronicles
• 2008 – The Curious Case of Benjamin Button