I wanted to share a little bit about how Google does code reviews, because both the infrastructure and the philosophy are potentially useful to developers in many companies.

Google tries to do as much as possible through the browser, and that extends to code reviews. We use a web-based code review tool called Mondrian that integrates with the rest of the development infrastructure to provide some unique features.

For example, as you revise files on your local disk over the course of a code review, Mondrian automatically snapshots each revision, without requiring any interaction with the source control system. Mondrian allows reviewers to diff between any two snapshots or the repository version of a file to be sure that all comments have been addressed. Reviewers can leave comments on any line of the file, and authors can respond back to the comments inline. Keyboard shortcuts (similar to those in Gmail) allow reviewers to quickly navigate through the differences and comments.

Mondrian’s features address some of the big issues I’ve encountered with code reviews in the past. Without having to sit down side by side in real time, it’s easy to do a good code review and ensure that all comments have been fully responded to. This makes geographically distributed development easier.

The key ideas behind Mondrian have been released in an open source project called Rietveld, which uses AppEngine and Subversion.

Mondrian was written by Guido van Rossum, author of the Python programming language and a Google employee. Guido has a tech talk that goes into some depth regarding the features of Mondrian.

The right tools enable more efficient engineering and better code quality. By reducing the friction of code reviews, Google can use them liberally. It’s typical to get a code review both from an owner of the package you are modifying and also (if you’re working in a language you’re not so familiar with) with someone who is certified in creating readable code in that language. A code review is a chance to both spot errors and to share expertise and ideas with other engineers.

Update: Curtis Bartley has an interesting post on strengths and weakeness of Mondrian and recommendations relative to Mozilla.

Update 2: One of the commenters asked about source control. Google uses Perforce as the primary source control system but a number of developers (myself included) use Git on their machines. Several Google developers have implemented a good bridge between the P4 and Git worlds as a 20% project.

I’m saying farewell to another old friend: the Beyond411 local search app for the Blackberry will discontinue service as of June 30th.

I first started working on Beyond411 (then called Berry 411) in July 2004. When I began, the Blackberry was the leading and best smartphone platform, and there were no good local search applications for it. It was gratifying to help fill that gap and to build an app that both I and my users appreciated.

Many versions and users later, the world is a very different place. There are a number of great search apps for smartphones. Smartphones have exploded in popularity and diversity, and I no longer even have a Blackberry.

It’s been increasingly hard to give Beyond411 the attention it needs. RIM is releasing an ever-increasing number of models and backwards compatibility has suffered; most have issues running Beyond411. I really enjoyed being an engineer at iLike and now Google but that leaves less spare hacking energy. Lastly, keeping the web service up and running requires a certain amount of ongoing attention.

Rather than giving users a bad experience, I’ve decided it’s best to give users time to find alternatives and then end the service. Thanks to everyone who has contributed your suggestions and support through the years!

I’ve just finished my first two weeks at Google as a “Noogler” and would like to share of my experiences here for those who are curious about Google’s approach to scalable engineering.

(One thing I won’t include is any description of food. I have a friend who has threatened to unfollow me if a yammer on about that topic!)

There is an amazing contrast between my experiences beginning as a new employee at a large Redmond company in 1995 and starting at Google today.

At that Redmond company, circa 1995, you had a day or so of orientation and then you were sent off to a particular team to begin work and learn the ropes. Each team had its own hiring bar, culture, coding standards, and ways of bring new employees up to speed. I was lucky enough to find my way eventually to a good team, but experiences certainly varied widely.

The informal, team-specific approach to onboarding new engineers has real problems as a company grows and scales out.

If they’re unlucky, new engineers may not get the help they need from their teammates and suffer getting up to speed. Engineers can’t easily move from team to team because the code and culture are so different, and teams can’t be certain that an engineer is qualified with re-interviewing them. Code reuse and knowledge sharing across teams is rare. Because of their different cultures and goals, teams sometimes develop an adversarial relationship.

Google has put as much thought into scaling their engineering team as they have scaling their systems; this leads to some unusual features in the way Google hires and rewards engineers.

You don’t interview for a specific team at Google, you interview to demonstrate that you can to do well anywhere in the company. This permits engineers to very easily move inside the company and provides for greater freedom.

Google has a formal engineering education program called engEDU. Every starting engineer learns how the most important parts of Google work (Search, Adwords, servers, datacenters, etc.); later there are classes on more advanced topics such as machine learning and distributed systems. The clases are taught by experienced engineers who really know what they’re talking about. There is a comprehensive set of ongoing classes, labs, and fully searchable set of wiki pages, created bottom up.

Transparency and knowledge sharing are prevalant and strongly encouraged by the culture and peer review system. Teams and individuals publish their goals, and quick status updates (called “snippets”) online to allow anyone in the company to read them. Food, social events, and tech talks all encourage serendipitous meetings across teams.

Though Google has many offices, all new engineers spend at least a week in Mountain View for training, to absorb the culture from fellow engineers and senior leadership, and to meet remote members of their team they will be collaborating with.

My experience so far has been that their Google’s culture and cohesiveness have scaled remarkably well in the face of rapid growth. I’d be happy to answer any questions you might have; just drop me an email.

(The following alternate history imagines what might have happened if Steve Jobs had joined Steve Ballmer in creating Windows. Any resemblance to reality is purely coincidental.)

Steve Jobs at Microsoft

Nearly everyone knows the story of how Steve Jobs and Steve Ballmer dropped out of Berkeley to build one of the most loved and feared products in history, Microsoft Windows. For those who don’t, we’ll recap the history of Windows.

Windows 1.0: Even at this early stage, Windows combined Ballmer’s business savvy with Jobs’ passion for design. Jobs rightly realized that other personal computers hadn’t caught on because they were too complex and too insecure, mostly because of unauthorized software.

The first version of Windows ran only a fixed set of programs, and didn’t support multitasking, But the built-in programs were so beautiful that no one cared too much; they included the mTunes media store, Internet Explorer web browser, and bundled Microsoft Office suite.

Ballmer signed Windows deals with Dell, IBM, and other PC makers, with strict provisions to ensure that Windows ran exactly as Jobs wanted it, and Windows rapidly gained market dominance.

Windows 2.0 introduced a revolutionary feature, the Windows App Store, allowing user to install approved programs from other software vendors. Microsoft took only 30% of revenues and gave 70% to the app makers.

The app store rapidly caught on, even though some software makers complained that Microsoft gave its own software an unfair advantage by using unpublished APIs and by rejecting competing software from the store.

The makers of an obscure Linux browser known as Netscape filed suit against Microsoft for rejecting their app, which Microsoft said was a security and stability risk. The judge rejected the suit, pointing out that no actual users had indicated a desire for any other browser. Lotus 1-2-3 also made no headway against Excel; their app was rejected for duplicating too much functionality from the built in Office suite.

Windows 3.0 delighted Windows fans by adding a cut and paste feature.

It also integrated the recently purchased search engine from Alta Vista. Users loved the convenience of being able to access Windows Web Search right from their Internet Explorer search bar.

Windows 4.0 remains the subject of much speculation, given Microsoft’s passion for secrecy. The addition of multitasking to Windows seems likely; some of the more outlandish rumors include a Windows ad network and even a Windows phone!

Some might argue that Jobs’ and Ballmer’s viselike grip over the Windows ecosystem gives them too much power. But most agree that protecting the purity and security of the beloved Windows user experience makes it all worth it.

I’m excited to be joining Google, starting April 5th. I’ll attend orientation week as a Noogler in Mountain View and work out of the Kirkland office.

Leaving the startup world is bittersweet, of course. Despite working at several different startups over the past 10 years, I’ve been lucky to work with a core set of trusted friends almost without interruption. (My high school friends and even my kids tease me about the number of companies I’ve worked for, but I’ve had more stability in coworkers than most folks.)

Compared with my big company experience at Microsoft, startups have less politics, less meetings, more opportunities to own a product end-to-end, greater impact on the failure or success of the business, and more exposure to company leaders and mentors.

On the other hand, there are limits to what you can learn working only in startups. Unless you get really lucky and discover that you’re working at the next Google, there are levels of scale and infrastructure that you won’t experience in a startup. Also, in the current economic climate, it seems difficult for truly innovative startups to achieve success.

Google is one of a very small set of companies that combine a lot of the things I about like startups with the opportunity to work at the highest level of scale and impact large numbers of people. One of my oldest friends from graduate school spent a lot of time helping to realize that Google cared about more than just IQ and valued the same things I do.

A number of folks have blogged about the experience of being a new Google employee; if I feel I have anything unique and disclosable to offer I’ll post it here. (As always, I am writing here as myself and any opinions expressed are not necessarily those of Google.)

I will immediately address one frequently asked question right away: “What will you be working on?” A certain amount of mystery surrounds that question, even for me.

Most folks at Google begin with a starter project, which gives them a chance to learn the key technologies of Google and investigate over time which team they’d like to join for the long term. (The starter project is an unusual approach and I first saw it as a negative factor, but as i learned more I came to see it as a good thing.)

My high level goals for a starter project were that I should be working on something that faces end users, that it should help me learn how to take advantage of Google’s infrastructure, and that it should be important to Google’s revenue. In advance of starting, I have just enough information to know that my initial project will meet these goals; I’ll say more as I learn more and am able to.

If you use a SOAP based API via the soap4r, you might find the added helper methods described in this post to be.. err.. helpful.

Without the helper, Soap objects are frustratingly opaque. They look like this when you inspect them:

And when you access attributes, you have to use ugly method names, for example e.xmlattr_venueName.

After, Soap objects look like this:

Much better! Plus attributes can accessed without the xmlattr_ prefix, and you can get a hash of all of the attributes.

soap_helper.rb is just a couple of dozen lines of Ruby. If it gets bigger, I'll put it up on Google code, but for now I'll include it inline in this blog post.

When building the Android version of the iLike Concert app, I wanted to have day headers for the concert list.

This was easy on the iPhone; sections are built into the oUITableView class.

Surprisingly, the ListView class in Android doesn't provide any corresponding functionality, as of Android 2.1.

Jeff Sharkey's SeparatedListAdapter provides one way to get this functionality; it glues together a number of individual adapters for each section into an overall list.

In my case, I had one SimpleCursorAdapter for the whole list, so I didn't want to break it into individual adapters just to display sections.

Instead, I made the header a part of the list cell layout, and hid the header on the rows where I didn't want to display it. The headers look as follows:

My layout was like as follows; note the use of the explicit background color and textColorHighlight in the date header to prevent it from appearing as selected.

The view binder assumes a function called isHeaderVisible to decide whether the header should be visible for the current row in the cursor; implement as appropriate for your application. (For example, our application displays a header whenever the date of the current row is different from the previous row.)

The local search app Beyond411 (aka Berry411) is now available as a free download for the iPhone in the app store, just search for the abbreviation "b411". This is just an initial version and doesn't have all of the features of the Blackberry version. Be forgiving of it's faults, and please pass on your bug reports and suggestions for improvement.

If Microsoft or Google found a way to predict and profit in the stock market by mining search logs, would it be a violation of their privacy policy?

This is not an outlandish scenario. The research paper Stock Prediction using Web Sentiment describes "a novel way to do stock prediction using web sentiment". The authors do textual analysis of financial message boards to extract how people are feeling about different stocks, correlate past sentiments with actual stock performance, and predict future stock values based on current sentiments.

The authors claim a strong correlation between web sentiment and future stock prices. Suppose for the sake of argument that you could actually predict and profit in the stock market with this strategy.

In principle, it seems that your ability to predict the stock market might be even better if you had access to the web search logs-- users would presumably reveal things in their private searches they wouldn't state publically. (To venture into even murkier privacy waters, consider also mining private emails.)

Google and Microsoft, of course, do have access to this data via their various web properties, and are growing increasingly sophisticated in their ability to mine this data.

They already analyze and expose aggregate search trends, for example at this moment Google Trends. At this instant Google Trends tells us that Copenhagen, Christmas Music, and Tiger Woods updates are on people's minds.

Suppose a talented Google engineer decided to create "Google Stock Trends" as a 20% project. I can't find anything in the Google privacy policy that would prevent that kind of aggregation-- it doesn't reveal any personally identifiable information.

Stock trend analysis seems to be included in the allowed purposes that Google makes use of personal information, including the vague "development of new services".

At the same time, it seems a violation of trust to think that others might profit from my revealing personal stock knowledge via a search engine query. I might not even know they were doing this mining, if Google Stock Trends was for internal use only.

Presumably a reputable company would actually never create such a tool. Should their privacy policies be more explicit that this kind of value extraction and data mining isn't allowed from searches that we think are private?

Stock Prediction

Skinny controller, fat model is a good design principle not just for Rails, but for client apps in MVC frameworks like Cocoa.

I think it's not uncommon, though, for client apps to have fairly anemic models, to the detriment of reusability. For example, I recently worked with an open source module that was in general very well written, but which made it challenging to reuse to underlying parsing logic.

Why this tendency for skinny models? In part, this might because client apps are UI-centric and this ends up being reflected in the design of the code. Even Apple's naming conventions (e.g. DetailViewController) emphasize the controller and view at the expense of the model.

The lack of a full-fledged read-eval-print loop in Objective C (akin to the Rails console) also makes it inconvenient for developers to exercise fat models. MacRuby has potential to provide an interactive console for Objective C models, and could perhaps be integrated with Xcode to allow me to bring up a console on any app.