Cashing In as Industry, Academia Mix
Last week, I went back to Soda Hall, the green-tiled architectural monstrosity that houses the computer science department at UC Berkeley. The building was filled with the same quiet hush and hurried, unkempt students I remembered from when I was a grad student there two years ago, but the atmosphere was palpably different. There was the distinct smell of money in the air.
Computer science departments, especially those near Silicon Valley, have always profited from close ties to industry, whether they be consulting gigs or equipment donations. But with the current climate of Internet hysteria, the opportunities for computer science professors and grad students to cash in have never been greater.
Already, a number of Berkeley grads have turned their expertise into Web riches, and there is now a subtle pressure on other students to do the same. “Sometimes I’m almost embarrassed that my research doesn’t have anything to do with the Web,†one grad student admitted to me. “I have to remind myself that I didn’t come to grad school to make money.â€
This commercial pressure isn’t necessarily a bad thing for the field. Computer science departments, after all, have long been criticized for turning out engineers unprepared for the real world of deadlines and budgets. On the other hand, one of the great strengths of academia is that it gives people the freedom to do far-out, blue-sky research. Losing some of that freedom could harm the pace of innovation.
To see the pros and cons of the growing blurring between academia and industry, you just need to look at how one research project at Berkeley--known as Networks of Workstations--has been affected.
When I was a grad student at Berkeley, the NOW project was the central research effort of the department. The idea was simple: Why not hook up a bunch of workstations with a fast network, then have them work together as if they were one giant supercomputer?
The notion came to be called hive computing, and it promised a huge advance over the specialized and expensive supercomputers of the day. But once the first prototype system was up and running, the group needed a sample application to demonstrate convincingly the capabilities of a NOW.
The application that Eric Brewer, a new assistant professor, and Paul Gaulthier, a grad student, dreamed up was a high-speed search engine for the Web. The search engines of the day were having a hard time keeping up with the rapid development of the Internet. Searching 40 million Web pages to find the official Thomas Pynchon site required more computational horsepower than a single workstation could provide. Thus was born Inktomi, a complete index of the Web that ran on a network of eight Sun workstations.
Normally, that would have been the end of the story. Maybe a few papers would have been written about the system and then it would have been left to languish. But this was at the very peak of Web mania, when companies such as Yahoo were going public at ridiculous valuations, and Brewer and Gaulthier were besieged by calls from venture capitalists interested in commercializing Inktomi.
They went for it with gusto. After a couple of rounds of financing, Inktomi Inc. is now a 30-person company with a commercial Web site up and running, complete with advertisements (https://www.hotbot.com). Inktomi is a success (at least for now), but there is plenty of rumbling within Berkeley about whether the company hurt or helped the NOW project.
More than anything, says Brewer in its defense, Inktomi has served as a reality check for the NOW project. “Trying to run a service that is always operational has really taught us a lot about what it takes to build 24-7 [24-hour, 7-day-a-week] systems†he told me. “Inktomi has also exposed a number of positive things about NOWs that we hadn’t previously considered. For example, because we can use multiple vendors--instead of having a single vendor, as with a conventional supercomputer--we get much better service.â€
Inktomi has also raised the visibility of the NOW project and given it added credibility. As evidence, Brewer cites a recent talk he gave about NOW at a prestigious technical conference. “I wouldn’t have been invited to give that talk if it wasn’t for Inktomi,†he points out. (Inktomi also has boosted Brewer’s career--no small thing given his current untenured status.)
But while most people I spoke to at Berkeley about Inktomi were quick to echo these benefits, they invariably asked one pointed question: “How many papers have you seen about Inktomi?â€
In a world still governed by the maxim “Publish or perish,†where the overriding goal is to share what you have learned, people bristle at the silence and nondisclosure agreements that come out of Inktomi Inc.
It’s a problem Brewer is quick to cop to, although with a couple of caveats. First, he says, the lack of published papers has more to do with lack of time than with an effort to protect trade secrets. Second, he argues that even on sensitive issues, publication will only be delayed.
“I don’t want our competition to find our secrets,†Brewer says, “but if I wait a year, then they will no longer be a competitive advantage, or else they will be patented and I can publish them.â€
Perhaps. The real test will come in a few years, when today’s grad students go out to become professors and engineers. Will the current intermingling between industry and academia result in a combination of the best traits of both? Or will graduates end up being both more secretive and less imaginative?
At least for now, Rich Martin, one of the grad students at Berkeley who saw his NOW research end up being used by Inktomi, believes the result has been positive.
“It gave the NOW project a lot of credibility, and that outweighs some of the disadvantages such as our inability to go over there and study how the code behaves in real-world conditions.â€
Besides, he adds with a touch of amazement, “with all the excitement about Inktomi, I’m starting to get calls from headhunters.â€
