Backblaze is a cloud backup service that needs cheap storage. Lots of it. They say a petabyte worth of raw drives runs under $100,000, but buying that much storage in products from major vendors easily costs over $1,000,000. So they built their own.

The result is a 4U rack-mounted Linux-based server that contains 67 terabytes at a material cost of $7,867, the bulk of which goes to purchase the drives themselves.

And best of all, they open sourced their hardware:

I’ve been a fan of Drobo since I got mine over a year ago. The little(-ish, and sweet looking, for stack of disks) device packs as many as four drives and automatically manages them to ensure the reliability of your data and easy expandability of the storage. However, Thomas Tomchak just pointed out one major flaw: if you overflow your Drobo with data, the entire device may give up and you’ll lose everything.

How do you overflow a Drobo? Most users only have a few terrabytes of storage in their Drobo, but configure it to tell the computer its attached to that it can store eight or 16 TB of data. Doing that allows easy expansion when more or larger drives are added — the attached computer doesn’t need to reformat anything, it can simply save more stuff to the device — but it also opens the door to the Drobo overflow.

From Tomchak’s post:

While on my tech support call I asked the engineer how frequently he received calls about this particular problem. After a big sigh he admitted that it was nearly every day.

One commenter on the article suggested the Drobo could “just simulate that the uninstalled part is already full of simulated read-only data,” a suggestion that makes sense, but may require the Drobo to know more about the filesystem on it than it otherwise would.

I’ve been at 90% capacity on my Drobo for a while, I think it’s time I popped another disk in there.

(CC licensed photo by Pixelthing.)

A few years ago I found an article pointing out how spammers had figured out how to abuse some code I wrote back in 2001 or so. I’d put it on the list to fix and even started a blog post so that I could take my lumps publicly.

Now I’ve rediscovered that draft post…and that I never fixed the bad code it had fingered. Worse, I’m no longer in a position to change the code.

Along similar lines, I’ve been told that a database driven DHCP config file generator that I wrote back in the late 1990s is still in use, and still suffers bugs due to my failure to sanitize MAC addresses that, being entered by humans, sometimes have errors.

I’ve written bad code since then and will write more bad code still, but as my participation in open source projects has increased, I’ve enjoyed the benefit of community examples and criticism. My work now is better for it.

Most of my work is available publicly, but some development is hosted on a private SVN that’s hidden behind a firewall. Unfortunately, my primary development server is on the wrong side of that particular firewall, so I use the following command to bridge the gap:

ssh -R 1980:svn_host:80 username@dev_server.com

That creates a reverse tunnel through my laptop to the SVN server and allows me to checkout code using the following:

http://localhost:1980/path/to/trunk

I’m posting that because I lost my terminal command history and had to think for a moment about how to do this again.

Years ago I used to tunnel my outgoing email to an un-authenticated SMTP server that only accepted outgoing messages from hosts on the local network. That was fairly common back in 2000 or so, but obviously made life (or communication) difficult for people at home or on the road. The easy solution was to SSH to a machine on mail server’s local network and forward emails through it.

ssh -L 1925:email_host:25 username@ssh_host

Doing that, I was able to configure my mail client to send outgoing emails using a server configuration like the following:

SMTP host: localhost
SMTP port: 1925

Search Engine Land.com:

Yelp…is…essentially a poster-child for semantic markup. This spring, Google’s introduction of rich snippets has allowed Yelp’s listings in the SERPs to stand out more, attracting consumers to click more due to the “bling” decorating the listings in the form of the star ratings.

There are now some very good reasons why sites with ratings and reviews should be adopting microformats, and it’s not that hard to do! For a more detailed explanation, read my recap on the subject, Why Use Microformats?

In March of this year Apple applied for a patent on technology that enables or disables features of a phone via a config file. The tech is already in use: it’s the carrier profiles we’ve been downloading recently. On the one hand this is just an extension of the parental controls that Apple has included in Mac OS X since the early days, but it also implies some rather anti-consumer thinking at the company.

One examplar claim in the patent is that the config file can include a “blacklist of device resources to be restricted from access.” 

AT&T used this this technology to block MMS until recently, and uses it now to block tethering, but the description given in the patent application goes much further:

For example, a carrier may wish to provide an enhanced service which utilizes the global positioning system (GPS) functionality in a mobile device. Carrier may wish to charge a premium for this service, so it may configure carrier provisioning profile to disallow third party applications from accessing the GPS functionality in device, and instead only allow applications digitally signed by carrier (or another entity affiliated with carrier) to access the GPS services in device.

Readers may remember the Trusted Computing video by Lutz Vogel and Benjamin Stephan that spotlighted the growing interest within the computing industry to impose new and artificial restrictions on the way we use the hardware and software we use daily.

A quick Google search of klaomta.com reveals more than a few people wondering why it’s iframed on their websites. The answer is that the site has been compromised.

Unfortunately for the fellow who asked me the question at WordCamp, solving the problem can be a bit of a chore. Keeping your WordPress installation up to date is important, as there are some known security flaws in older versions, but most of the attacks that crackers use are targeted elsewhere. Your passwords, all your server apps, the PHP config, your hosting control panel, and other users all must go under the microscope when trying to find security holes.

Plugin Development

Will Norris‘ talk at WordCamp PDX introduces WordPress coding standards, common functions, and constants to would be plugin developers (and smacks those who’ve already done it wrong). Also notable: functions, classes, variables, and constants in the WordPress trunk.

Custom Installations

Just as WordPress has a number of hooks and filters that plugins can use to modify and extend behavior, it also has a cool way to customize the installation process.

Extending The WYSIWYG Editor

TinyMCE, the WYSIWYG editor in WordPress has a rich API to allow adding buttons and stuff, but the docs are hard to get into. We can get a jump on that by looking at how it’s implemented in other WP plugins. This code creates the buttons, while the function that responds to the button click and does the work is defined within the plugin. The TinyMCE plugins in core are also informative.

The one way relationship between our user accounts and the MIS database also makes it difficult to engage with new users online. If you can’t get an account until you become a student, how do you allow potential students to apply online if all your systems are integrated with single sign-on? And if you can’t authenticate the online identity of your users, how do you set initial passwords into your system? Or allow them to reset a forgotten password online?

Internet companies never struggled with this issue, as their customers could only approach them online, but most universities built systems around paper applications and have fond (and relatively recent) memories of offering their students their first internet experience. It’s still not unusual for universities to offer their students their campus computing account with a default password based on supposedly secret data shared between the user and the school. But your SSN, birth date, and mother’s name are no longer secret. A proposed change in FERPA policy (see the the top of page 15586 in the NPRM) would have barred the use of “a common form user name (e.g., last name and first name initial) with date of birth or SSN, or a portion of the SSN, as an initial password to be changed upon first use of the system” in systems that store academic data. The final rule excluded that provision, much to the relief of those schools with more lobbying clout than brains.

Platform Choices

Rather than wait to see how the ruling played out last year, we went to work trying to improve security while easing access to our systems (no, that is not self-contradictory). Our challenges were thus:

• Fix initial password assignment
• Fix password resets
• Allow users with a loose or undefined relationship to the institution to create limited accounts for the purpose of interacting with the institution or its members

We considered a number of paths to a solution, including hacking of our university portal (which hosts the CAS single sign-on in our environment), expansion of a limited home-built solution, and a review of commercial and open source products and frameworks. We simplified the problem by confirming that the FERPA rule did not require us to authenticate the “real life” identity of a person; rather, we had only to validate the online identity of a person (saving us from needing to do things like send confirmation PINs by postal mail to a person’s home address).

In the end, we chose WordPress MU. Significant factors were our experience with the software (all the MIS developers use it personally), the extensibility of it as an application platform, the development focus on user experience (especially in recent versions), and our interest in using it as a framework for other user-facing services (especially BuddyPress).

Our Needs vs. WordPress

• The system must serve as the front end to our single sign-on environment, using our AD and LDAP password stores to authenticate users who have accounts in those systems.
• External email addresses, once verified with some challenge/response, can be used to reset a password.
• Users who are presently affiliated with the school have a school-provided email address, but no external address with which to reset their lost passwords.
• Users who are not presently affiliated with the school have no school-provided email address, and must verify their external email address before their account is activated. They can then set their own password once they verify their email address.
• The ability to send password reset codes via SMS would be nice (especially considering the number of long-time employees of the university who do not have personal email accounts), though that also requires the verification of the user’s cell phone number.

After reviewing what we wanted to do, we surveyed WordPress’ code to develop an implementation plan. And, because a number of aspects of our application process were changing, we decided to focus on allowing current users to self-reset their password and postpone development of account self-creation features for new users. Still, a few issues quickly emerged:

• WordPress requires a username be assigned to each user, rather than relying on email address (this is likely to change in WP 2.9) Creating a new username for our users is unacceptable, but adding a large number of new users to our existing username space will quickly deplete the “good” usernames. And changing a user’s username as their affiliation with the institution is unacceptable.
• The core user authentication function can be replaced with our own function. (And in 2.8 it became filterable)
• WordPress MU will validate email addresses, but the system isn’t built to be extensible.
• WordPress only stores one email address per user, but the user meta system can be used to store a second one. Unfortunately (and in a manner inconsistent with post meta), only one value per meta key per user is allowed, making it difficult to allow users to have an arbitrary number of email addresses associated with their account.
• The function that identifies a user by a given email address can be replaced with a function that also checks the secondary address.
• WordPress user profiles have no phone field, but the user meta system can be used to store one. A function to identify a user by a given phone number must also be created.
• Unlike some settings pages, the fields on the user profile editor cannot be changed simply by modifying the $wp_settings_fields array.
• Upon doing a password reset, the user is sent a temporary password, rather than being allowed to set a new password. This contradicts University policy about how passwords are used and communicated and could train users that sending passwords by mail is acceptable.
• The various functions in wp-login.php cannot be replaced, and in WP 2.7 the code had no way to add or replace various login actions (WP 2.8 changed that).
• WPMU-specific functions don’t always follow WP coding standards or models.

(Note that we began our work and deployed the system under WPMU 2.7. WPMU 2.8 included a few changes that made the process easier. I’m proud to say that some of those changes were a result of code we offered back to WP during our development.)

What We Did

• We decided that email addresses (both PSU addresses and external addresses), as well as PSU usernames would be acceptable identifiers for an account, and that a person should be able to log in to our web services using any of those identifiers. So…
• We replaced wp_autenticate() with our own function that accepts either email address or university username, checks to see if the user exists locally, checks to see if they exist in AD or LDAP, confirms their password, provisions their WordPress account (for university users who’ve not logged in via this method yet), establishes a session with our university portal and redirects them there (unless $redirect is set to something more specific that the dashboard).
• We decided to replace WordPress’ usernames with a random string matching a pattern we established. This became the WPID. Doing this required us to hide references to username (easy if you set a preferred display name)
• To store phone numbers and secondary email addresses, and allow users to edit those within their profile, I created the Alternate Contact Info plugin (browse source). This requires more use of output buffering than I’d like, but it gets the job done.
• To confirm email addresses and phone numbers via a challenge/response message (and support other interactions), I created the WordPress Ticket Framework plugin (my introduction, browse source).
• To send messages via SMS, we used my wpSMS plugin (in the plugin directory, browse source).
• Matther Batchelder re-skinned the login screen via a plugin that inserts our custom CSS.
• After determining that our university portal could not be made to authenticate via CAS, I gave up work on my wpCAS Server plugin and developed another method to initiate the portal session (which then establishes a CAS session using the portal’s CAS server).
• We replaced most of the functionality of the wp-login.php page (by hacking core at first, then taking advantage of the action hook in 2.8). In doing so we were able to change the password reset behavior to allow users to immediately change their password after entering their reset code (which was sent to their email address or phone via SMS).

Over time we extended the system to host multiple domains and replace our CMS. Soon we’ll consolidate our  public blogging instance into it, and we’re building an invite system that we can use to invite people to join our community.

What It Looks Like

The re-skinned WordPress login

Entering an email address or username to get a password reset code

SMS text with password reset code

Enter the password reset code from the SMS text message here, or follow the link from the email

Extended contact information in the WordPress profile

And that’s how we replaced our authentication system with WordPress, gained self-service password resets, and built the foundation to invite new users into our system.

Moving data by homing pigeon requires some planning, and pigeons

It was a tech story so apparently humorous that the popular media felt compelled to cover it: carrier pigeons delivered 4GBs of data faster than an ADSL line. The BBC story’s subtitle read “broadband promised to unite the world with super-fast data delivery – but in South Africa it seems the web is still no faster than a humble pigeon,” and that’s how most stories played it. Unfortunately, they all got it wrong.

The race was run by The Unlimited Group, but the clearest telling of it comes from Wikipedia:

Inspired by RFC 2549, on 9 September 2009 the marketing team of The Unlimited, a regional company in South Africa, decided to host a tongue-in-cheek “Pigeon Race” between their pet pigeon “Winston” and local telecom company Telkom SA. The race is to send 4 gigabytes of data from Howick to Hillcrest, approximately 60 km apart. The pigeon carrying a microSD card (an avian variant of a sneakernet), versus a Telkom ADSL line. Winston beat the data transfer over Telkom’s ADSL line, with a total time of two hours, six minutes and 57 seconds from uploading data on the microSD card to completion of download from card. At the time of Winston’s victory, the 4GB ADSL transfer was just under 4% complete.

The critical numbers here are 4GB vs. 164MB (4% of 4GB) in 2 hours. Also, because the challenge was peer to peer transfer, rather than a simple download, the bandwidth is limited by both the upload and download speed of the technology. ADSL, like most consumer broadband technologies, offers slower upload speeds than download speeds.

Speedtest.net's summary of worldwide test results

My home ADSL line is claimed to be 3Mbps/768Kbps, but manages a little less than that in practice according to Speedtest.net (and their sweet iPhone app). That’s a little below average for the US (be sure to view upload speeds), but not dramatically so. And even the fastest region, Asia, can only boast 1.72 Mbps.

In the race between pigeon and internet, the 164MB network transfer over 2 hours is about 187 Kb/s. Significantly slower than my home connection is rated for. But even the fastest consumer upload speeds would have trouble beating the pigeon’s transfer speed. It would take sustained uploads of about 5Gb/s to transfer 4GB of data in the two hours of the pigeon stunt, a rate much faster than possible with the company’s 1Mb/s ADSL connection. But it isn’t just the South African internet that’s slow: The best performing “ISP” in New Hampshire, Dartmouth College, gets a 10 Mb/s rating in Speedtest.net, though very few others can manage over 4Mb/s (Comcast advertises upload speeds up to 10Mb/s, though that’s with “PowerBoost”; sustained speeds are lower, the company reportedly throttles FTP and other uploads, and Speedtest.net rates their business-class uploads at 3.47 Mb/s). If the story makes South African networks laughable, it makes the whole internet a comedy act.

microSD card sizes compared to other SD types

And the pigeon could have carried even more data and made a bigger win over broadband. microSD cards such as the one carried by the pigeon weigh less than half a gram and are now available in capacities up to 16GB (with a theoretical capacity of about 128GB). Assuming you can attached two such cards (one to each leg), you’d get an upload rate of about 40Mb/s in the South African test. Perhaps you could even attach four such cards (totaling under 2 grams of cargo and yielding 80Mb/s) or send more pigeons.

Consumer download speeds are typically much faster, and popular web services go to great lengths to ensure they fill those pipes to speedily deliver rich media. Apple was an early investor in Akamai, one of the first content delivery networks that speed downloads by distributing servers around the world so that your iTunes music and movie downloads are delivered from the closest servers and through the fastest pipes to your location. Netflix’s on demand service leverages the broad availability of these services to deliver TV shows and movies faster than the US Postal Service. But comparing internet speeds to postal service delivery of DVDs (a 14 year old standard) gives no real suggestion of attainable postal service bandwidth (especially when the network-delivered content is far more compressed than on DVD).

You could probably stuff about 30 MicroSD cards (which have almost five times the storage density of regular SD cards — 32GB in 1612.8mm³ vs 16GB in 165mm³) in a First Class letter ($.44 for 6.125 inch x 11.5 inch and 1 ounce) that gets delivered almost anywhere in the US in three days, yielding about 480GB in 72 hours or 15 Mb/s. A pair of 512GB SSDs, on the other hand, can be delivered over night, yielding 1TB in 24 hours or 97 Mb/s. Given that DVDs are 1.2mm thick and 120 mm in diameter, Netflix could easily fit as many as 80 microSD cards — 1280 GB (41 Mb/s over three days) — in their current delivery envelope, much more capacity than even the 50 GB (1.6 MB/s over three days) capacity of the largest Blue-ray discs. It’s for that reason that Amazon offers bulk import/export services of physical media for their web services.

Point: storage capacity has increased dramatically over the past few years, while internet speeds have remained relatively stable. The pigeon gimmick didn’t show us a laughably slow internet connection, it showed us how unimaginably vast our storage has grown (and yet we still find ways to fill it).

Photos: Winston, Speedtest.net, Wikipedia. History worth remembering: avian photographers (via Modern Mechanix).

Ben Fisherman’s JSNES runs entirely in the browser using nothing more intrusive than JavaScript. It apparently manages real-time performance within Chrome, but it works (if not playably) on an iPhone.

I wish the screen was resizable and that it supported iPhone compatible controls, but both of those assume that browser performance will improve enough to make it playable. Interestingly, though not surprisingly, the Safari JS engine is limited to consuming a single CPU (which it quickly does while playing JSNES).

Will Norris has discovered that iTunes 9’s interactions with the Store are more web-happy. I’ve been asking where the iTunes Store API was for some time, now I think I’ve got what I need to build one.

• site.org/blogname1
• site.org/departments/blogname2
• site.org/departments/blogname3
• site.org/services/blogname3

The association between blog IDs and sub-directory paths is determined in wpmu-settings.php, but the code there knows nothing about nested paths. So a person planning to use WordPress MU as a CMS must either flatten his/her information architecture, or do some hacking.

Challenge: hacking WordPress MU to support arbitrary directory paths for each blog

As with my multi-domain hack, the following assumes that you’re using the vhost=no setting, that you have access to and know how to manipulate your MySQL, that you have control over your DNS and know how to use it, and that you know how to configure Apache or similar. You’d also be smart to turn off any object caching you may have running, at least until we’re done doing direct database manipulation. The following also assumes that your wp-config.php sets the DOMAIN_CURRENT_SITE and PATH_CURRENT_SITE constants — if you’ve done a fresh install recently, it probably does, or you can check my domain mapping hack.

Hack The Path Mapping

Right at the top of wpmu-settings.php you can see how it strips all but the base of the URL path, but rather than mod that file, we can take advantage of an obscure MU hack: sunrise.php, which gets executed after some important WordPress components like the database class get loaded and before wpmu-settings.php.

To use sunrise.php, create a PHP file at /wp-content/sunrise.php and set define('SUNRISE', TRUE); in your wp-config.php.

Here’s the sunrise.php code I’m using:

if( defined( 'DOMAIN_CURRENT_SITE' ) && defined( 'PATH_CURRENT_SITE' ) ) {
	$current_site->id = (defined( 'SITE_ID_CURRENT_SITE' ) ? constant('SITE_ID_CURRENT_SITE') : 1);
	$current_site->domain = $domain = DOMAIN_CURRENT_SITE;
	$current_site->path  = $path = PATH_CURRENT_SITE;
	if( defined( 'BLOGID_CURRENT_SITE' ) )
		$current_site->blog_id = BLOGID_CURRENT_SITE;

	$url = parse_url( $_SERVER['REQUEST_URI'], PHP_URL_PATH );

	$patharray = (array) explode( '/', trim( $url, '/' ));
	$blogsearch = '';
	if( count( $patharray )){
		foreach( $patharray as $pathpart ){
			$pathsearch .= '/'. $pathpart;
			$blogsearch .= $wpdb->prepare(" OR (domain = %s AND path = %s) ", $domain, $pathsearch .'/' );
		}
	}

	$current_blog = $wpdb->get_row( $wpdb->prepare("SELECT *, LENGTH( path ) as pathlen FROM $wpdb->blogs WHERE domain = %s AND path = '/'", $domain, $path) . $blogsearch .'ORDER BY pathlen DESC LIMIT 1');

	$blog_id = $current_blog->blog_id;
	$public  = $current_blog->public;
	$site_id = $current_blog->site_id;
	$current_site = sl_get_current_site_name( $current_site );
}

function sl_get_current_site_name( $current_site ) {
	global $wpdb;
	$current_site->site_name = wp_cache_get( $current_site->id . ':current_site_name', "site-options" );
	if ( !$current_site->site_name ) {
		$current_site->site_name = $wpdb->get_var( $wpdb->prepare( "SELECT meta_value FROM $wpdb->sitemeta WHERE site_id = %d AND meta_key = 'site_name'", $current_site->id ) );
		if( $current_site->site_name == null )
			$current_site->site_name = ucfirst( $current_site->domain );
		wp_cache_set( $current_site->id . ':current_site_name', $current_site->site_name, 'site-options');
	}
	return $current_site;
}

The first few lines of the code do pretty much the same as the start of the wpmu_current_site() function in wpmu-settings.php, but starting with line 8 it takes a big departure.

That’s where it splits the requested URL path like /path/to/blog/and/stuff/ into pieces and constructs an SQL query against the wp_blogs table to identify the correct blog to serve the request. The following example shows how:

SELECT *, LENGTH( path ) as pathlen
	 FROM wp_blogs
	 WHERE domain = 'domain.org' AND path = '/'"
	  	 OR (domain = 'domain.org' AND path = '/path/')
	 	 OR (domain = 'domain.org' AND path = '/path/to/')
	 	 OR (domain = 'domain.org' AND path = '/path/to/blog/')
	 	 OR (domain = 'domain.org' AND path = '/path/to/blog/and/')
	 	 OR (domain = 'domain.org' AND path = '/path/to/blog/and/stuff/')
	 ORDER BY pathlen DESC
	 LIMIT 1

Optimization note

Setting a maximum depth (and array_slice( $patharray, 0, $maxdepth )) would allow the query to be cached up to that depth. Otherwise, the query must be executed for every page load. The $maxdepth could either be set arbitrarily, or could be determined automatically based on the maximum path length of registered blogs.

Setting Up New Blogs

Once you’ve hacked the path mapping (and tested that it didn’t break your current site), you can add a new blog at a nested path.

Create a new blog in the MU blog admin.

Unfortunately, MU strips the slashes from the URL path you just tried to set.

The new blog you just tried to create, but with a very different path.

Fortunately, you can set the path correctly in the MU blog editor, and it won’t break the path when you save there.

Set the blog path in the MU blog editor, MU won't break it when you save it this time.

Once you create the new blog, try to load it in your browser. You’ll quickly notice the stylesheet is missing, though the blog works and functions properly.

Hack The .htaccess

WPMU uses the following .htaccess rewrite rule to properly direct requests for files on the real filesystem:

RewriteRule  ^([_0-9a-zA-Z-]+/)?(wp-.*) $2 [L]

Obviously, that rule won’t work for deep paths, so I’ve replaced it with this rule:

RewriteRule  ^(.+)?/(wp-.*) /$2 [L]

And with that, you should be done.

With AT&T facing lawsuits for not delivering MMS features at the iPhone 3GS launch, they kind of had to do something. I’m not sure if I’d be satisfied by this video if I were among the plaintiffs, but I think it does a good enough job. The stat about 300% annual increases in mobile data use is pretty powerful. I’d heard it a dozen times before*, but because I wasn’t in Austin for SXSW iPhone meltdown, I don’t have quite the same appreciation as some do. AT&T added capacity then, and they seem to have been scrambling elsewhere too.

iPhone users are said to be six times as likely as anybody else to watch video on their phones, and if WiFi aggregator JiWire’s report says anything about cell data, the iPhone has certainly changed the game. JiWire’s Mobile Audience Insights Report shows that over 97% of the devices on their network are either iPhones (about 56% of the total) or iPod Touches! And all the way back in 2007 in Britain, iPhone users were 33 times as likely as other phone users to send or receive more than 25MB a month.

It will be interesting to see what happens to other carriers as they get devices that encourage use as the iPhone has.

*Actually, I hadn’t heard the 300% stat specifically, just inspecific reports of increased usage.

Okay, I don’t want to watch all the movies depicted in this 100 year overview of film special effects, but I did just add a few to my Netflix queue.

The full list, according to the description in YouTube:

• 1900 – The Enchanted Drawing
• 1903 – The Great Train Robbery
• 1923 – The Ten Commandments (Silent)
• 1927 – Sunrise
• 1933 – King Kong
• 1939 – The Wizard of Oz
• 1940 – The Thief of Bagdad
• 1954 – 20,000 Leagues Under the Sea
• 1956 – Forbidden Planet
• 1963 – Jason and the Argonauts
• 1964 – Mary Poppins
• 1977 – Star Wars
• 1982 – Tron
• 1985 – Back to the Future
• 1988 – Who Framed Roger Rabbit
• 1989 – The Abyss
• 1991 – Terminator 2: Judgement Day
• 1992 – The Young Indiana Jones Chronicles
• 1993 – Jurassic Park
• 2004 – Spider-Man 2
• 2005 – King Kong
• 2006 – Pirates of the Caribbean: Dead Man’s Chest
• 2007 – Pirates of the Caribbean: At World’s End
• 2007 – The Golden Compass
• 2008 – The Spiderwick Chronicles
• 2008 – The Curious Case of Benjamin Button

WordPress MU is a solid CMS to support a large organization. Each individual blog has its own place in the organization’s URL scheme (www.site.org/blogname), and each blog can have its own administrators and other users. Groups of blogs in WPMU make up a “Site” and one or more Sites can be hosted with a single implementation. (I’m capitalizing Site for the same reason WordPress docs capitalize Page) Each Site has a defined set of administrators and options controlling various features. You might, for instance, lock down the plugins on your blogs.site.org, while keeping it open on your www.site.org. Or maybe you’d like to let your helpdesk staff create new blogs at blogs.site.org, but not at www.site.org. That’s what WPMU’s notion of Site can help you control.

Challenge: setting up service on multiple (sub-) domains

WordPress MU makes it easy to host both blogs.site.org and www.site.org within a single implementation, but there’s little documentation for how to do it.

Once you get WPMU up and running on one of your domains you can add another. The following assumes that you’re using the vhost=no setting (correct: I really mean vhost=no), that you have access to and know how to manipulate your MySQL, that you have control over your DNS and know how to use it, and that you know how to configure Apache or similar. You’d also be smart to turn off any object caching you may have running, at least until we’re done doing direct database manipulation.

Some might ask why I’m not simply using Donncha’s plugin, and the answer is simple: it only works for vhost=yes sites. For my own use, I find that sub-directories are easier for users to make sense of (go ahead, try to tell your mom to go to “myblog.sub.domain.org“).

Set up your web server

If you’re using Apache, you’ll either need to create an IP-based virtual host or manually configure your name-based virtual host for every (sub-)domain you plan to serve. Why: WordPress will handle the domain mapping for you, so it’s better to keep Apache out of the way and let WPMU own the entire IP.

Set up your DNS

Point each subdomain you plan to host in WordPress to your webserver. You can use a wildcard domain, but you don’t have to.

Create a new blog in WPMU

It doesn’t matter what you call it or what the path is, just create one. Now go edit it in the Site Admin:

Change the domain and path to match your new domain.

One quirk of WPMU is that it strips “www” from any domain name you enter (or is requested), so don’t bother trying to enter it (unless you’re willing to do some hacking to make it work). WPMU stores domain and path information in three locations: the wp_#_options table for the blog, the wp_blogs table, and the wp_sites table. When you edit a blog in the Site Admin, you’ll get a chance to edit the domain and path for both the wp_#_options and the wp_blogs tables. Clicking the helpful checkbox above will do most of that for you, but you’ll need to manually update the Upload Path.

Now make sure the new domain is shown in the blog's options as well.

You might be able to load the blog at the new URL as soon as you update those settings, but recent versions of WPMU set some constants in wp-config.php that can get in your way.

Reconfigure wp-config.php

Your wp-config.php might have something like this:

$base = '/';
define('DOMAIN_CURRENT_SITE', 'sub.site.org' );
define('PATH_CURRENT_SITE', '/' );
define('SITE_ID_CURRENT_SITE', 1);
define('BLOGID_CURRENT_SITE', '1' );

Those constants override the database checking that goes on in wpmu-settings.php to map the requested domain to a site. You have three choices: leave it as it is (and use only one “Site”), remove it and have WPMU do the mapping against the database, or expand the hard-coded mapping to include other sites.

I’ve used code like the following to do just that:

if( preg_match( '/(.+?\.)?([^\.]+?)\.site.org/i', 'www'. $_SERVER['SERVER_NAME'] , $matchedsubdomains ))
{
	switch( array_pop( $matchedsubdomains )){
		case 'connect':
			define('DOMAIN_CURRENT_SITE', 'blogs.site.org' );
			define('PATH_CURRENT_SITE', '/' );
			define('BLOGID_CURRENT_SITE', '1' );
			break;
		case 'www':
		default:
			define('DOMAIN_CURRENT_SITE', 'site.org' );
			define('PATH_CURRENT_SITE', '/' );
			define('BLOGID_CURRENT_SITE', '2' );
			break;
}
}
else
{
	define('DOMAIN_CURRENT_SITE', 'site.org' );
	define('PATH_CURRENT_SITE', '/' );
	define('BLOGID_CURRENT_SITE', '2' );
}

Set up your new Site

Once you have your new sub-domain working with one blog, you can create your new Site. Even if you don’t plan to create separate management policies for the different sites, it’s easier to create new blogs at each sub-domain if they each have their own Site.

Go in to your MySQL tool of choice and browse the wp_site table. There you’ll see just one row, but if you’ve made it this far you can also probably figure out how to create a new row representing the site at the new sub-domain. And once you do that, you can change the entry in the wp_blogs table to associate it with your new Site.

Set up the admins of the new site

Creating the new entry in wp_site doesn’t set the options for the new Site, and that means there are no Site administrators yet. Once again in your MySQL tool of choice, open up the wp_sitemeta table and look for an entry with meta_key = 'site_admins'. The meta_value for that entry is a serialized array containing WordPress usernames of the people who have site-wide administration privileges on the first Site. I’m assuming that if you have MySQL access you’re also a Site admin, so the easy thing to do is copy that row and change the site_id to match the auto-increment value from the new wp_site entry you made in the last step.

With the database manipulation done, you should now be able to go to the WP dashboard at your new Site, visit the WPMU admin options screen, and set the other options as necessary. You could decide to make one of your Sites open registration (remember, however, that users are shared across all Sites), while making other Sites more closed. And, obviously, you can delegate different Site admins for each Site.

Sub-domains or just different domains?

It’s worth noting that the instructions so far apply to both sub-domains and domains. You can use a single implementation of WPMU to manage content at both lolzors.org and tehsite.org. The detail about sub-domains really only applies to the next part. It’s also worth noting that you can support an arbitrary number of blogs, sites, and domains; I’m just using two sub-domains as an example.

Challenge: unified log in cookies

What’s the point of hosting multiple sub-domains with one WPMU implementation if you’ll need to log in separately at each one?

Set your cookie path

Setting a cookie path that that’s broad enough to cover the entire domain will solve this:

define('COOKIE_DOMAIN', 'site.org');
define('ADMIN_COOKIE_PATH', '/');
define('COOKIEPATH', '/');
define('SITECOOKIEPATH', '/');

Avoid conflicts with other WordPress installations at your domain

But having such a broad cookie domain can interfere with other WordPress implementations. You’ll have to solve that by setting a unique cookiehash:

define( 'COOKIEHASH', 'asdf_arbitrary_string' );

You’d do better to keep it shorter than that, though.

Challenge: unified log in location/URL

WordPress MU is happy to handle authentication requests wherever it hosts a blog, but some organizations prefer to funnel all authentication requests through a single location. The idea is to provide some protection against fishing (assuming users can ever be taught to look at URLs) and make it easer to integrate external applications.

Filter login_url and logout_url

Set the log in and log out path to whatever you want, just make sure the destination knows how to create (or destroy) the WordPress cookies.

$hack_base_domain = 'site.org';

function hack_login_url( $path ){
	global $hack_base_domain;

	return preg_replace( '/^.+?\/wp-login.php/' , 'https://login.'. $hack_base_domain .'/wp-login.php', $path );
}
add_filter( 'login_url' , 'hack_login_url' , 10 );
add_filter( 'logout_url' , 'hack_login_url' , 10 );

Filter allowed_redirect_hosts

WP will normally block redirects outside the web root of the active blog, so you’ll need to tell it about your other sub-domains.

function hack_allowed_redirect_hosts( $allowed_domains ){
	global $hack_base_domain;

	$allowed_domains[] = $hack_base_domain;
	$allowed_domains[] = 'www.'. $hack_base_domain;
	$allowed_domains[] = 'blogs.'. $hack_base_domain;

	return $allowed_domains;
}
add_filter( 'allowed_redirect_hosts' , 'hack_allowed_redirect_hosts' , 10 );

I don’t know if it’s just the Mother’s day effect, but the top 10 online retailers for May 2009 were dominated by flower shops. The top shop is converting almost 40% of their visitors to buyers, though the average is just over 5%. Tim, meanwhile, claims he’s lowered his bounce rate to just 10%.

Liam Lynch explains the origin of the video, but what was Dan Deacon thinking as he recorded the audio? Of all the free MP3 downloads he offers, Two Friends from the Acorn Master album may be the most, um, listenable.

Thanks to daily songsmith Corey B (Corey Blanchette) for the tip.

The following was my email response to a thread on the web4lib mail list:

Okay, it must be said: you’re all wrong[1].

I can understand that news of a librarian being fired/furloughed will raise our defenses, but that’s no excuse for giving up the considered and critical thinking that this occasion demands.

Consider this: the principle’s blog reveals a reasonable person actively trying to improve academic performance despite crushing economic conditions. The communications show a level of transparency many of us can only wish for.

Unfortunately, many on this mail list seem to have come to the conclusion that this library was a stellar, but unappreciated example of everything that libraries should be, capriciously closed by a principle who secretly wanted to see the football team shoving bookshelves around on a hot summer day.

Go ahead, mock the story (and so far we only have one story about this) for suggesting that the books have been “re-organized” by subject, but the fact remains that this community didn’t think their library was organized in a way that met its needs. This suggests that either (a) it wasn’t well organized, or (b) the librarian had failed to educate the users and develop the finding aids necessary to help the community use the library.

Nobody here is banning or burning books. Nobody is suggesting that libraries are irrelevant. Far from it: this story about the modernization of a library to make it a more significant part of students’ academic activity.

News that a member of our profession has been furloughed is sad. But, news that a principle is investing time, attention, and money in the library is good. News that those two stories are one in the same should make us ask critical questions about how we and our libraries are positioned to serve our community.

[1]: Everybody but Robert L. Balliot, whose message has so far been ignored.

An amusing hacks-conference lightning talk-turned-blog post on web development: “Graceful Hacks” – UX, IA and interaction design tips for hack days. Martin Belam’s talk at The Guardian’s July 2009 Hack Day must have been both funny and useful:

• Funny: “However, I am given to understand that this is now deprecated and has gone out of fashion.”
• Useful: “the Yahoo! Design Pattern Library is your friend.”

The AP’s diagram of their Protect, Point, Pay “news DRM” scheme looked like a joke, then I saw the parody.

Despite all the smoke and hype, Ed Felton explains that it’s underwhelming, at most. Still, hNews might be an interesting format for some blogs to adopt. Most of what the AP is rattling their saber about is in the rights (containing ccREL declarations). Felton thinks the dependence on ccREL may extend derivative usage rights, rather than limit them. ccREL, after all, “states unequivocally that it does not limit users’ rights already granted by copyright and can only convey further rights to the user.”

Okay, so hNews mightn’t be so bad, but what’s good about it? It brings together a number of pieces that we all expect in a news story (and many other stories). It makes it easy to identify the dateline and geocoding of a particular story, as well as the publisher and its principles.

Oddly, the format doesn’t appear to address media within the content, but perhaps they expect us to leverage Media RSS and rel=image_src links.

All Facebook is happy to share the ten laws of Facebook advertising, but will those rules lead to better results than the .02% CTR Bob Gilbreath got a year ago?

Howard Weaver wants newspapers to play offense against Google and others, but Chris Tolles, CEO of news aggregator Topix.com says he’s been trying Weaver’s plan for a while, and there’s no bucket of gold to be found in it.

The problem, it would appear, is that newspapers don’t sell news. They sell advertising space and pair it with news as a loss leader to keep the eyeballs. And while that worked in print, it doesn’t work on the web. Web users don’t click on ads on news sites (perhaps for the same reasons they don’t click ads on social network sites).

Yes. The news in a newspaper was a loss leader, though it’ll probably make you feel better to think of the sports and entertainment that way. Television overtook newspapers as the primary news source in the 1970s. And in 2008, newspapers fell behind internet sources. The economics of it now are that news printed on bad paper are less of a loss leader and more simply just a loss. (It’s hard to know if the poor performance of newspaper websites is a result of this or if it reflects other failings.)

Those worried about what will happen to the public sphere after the death of newspapers should take a look at this 2004 study comparing television vs. newspaper front pages: newspapers had twice as many lifestyle and crime stories as the evening TV news. The public sphere has long needed better nourishment.

Mashable’s Josh Catone thinks NPR is the future of mainstream media. Their audience is way up, and it’s a well proven example of the crowdfunding model. My local affiliate provides some of the best local news around.

Google’s own webmasters help site recommends microformats and RDFa structured data to improve indexing and usefulness of the data. Review metadata appears to have full support, while people, product, and business data are in beta.