The Cloud

I was reading Marc Andreessen’s blog entry about The three kinds of platforms you meet on the Internet. and all his talk about sharing code and running that code on other systems and community and all that really got me thinking.

There have been many large distributed projects that have promoted the idea of sharing your CPU cycles. SETI@Home is of course the most well known, but others have followed in their footsteps. Some of these groups have met with success for their project, and their problem got solved, others were never able to obtain the necessary adoption rate. I have CPU cycles to share, but I’m not willing to goto the trouble of seeking out a project to contribute to and then downloading and installing their code so I can help solve their problem.

What’s needed here is something more akin to Bittorrent, where each researcher in need of a distributed parallel processing machine can ssh into ‘the cloud’ and then program up their problem, and wait for the answer. Just as if they’d logged into one of the many (expensive) super-computing facilities available to professional researchers. Now I know that high-performance computing has suddenly had a sharp drop in price, but that’s not really enough, that is it’s not wide-spread, and it’s still difficult to achieve. I would be willing, and any researcher would probably be similarly inclined to donate cpu cycles to another cause. It’s just that we’re not gonna go out of our way to do so, unless we get something in return. I would gladly run a daemon on my system which accepted jobs for another researchers problem, and sent back results, if I could get the same type of treatment for my problems. This altruism is especially true if the daemon was a VM that supported multiple languages (or even a specialized distributed parallel language) that had a shell-like or ssh-style interface.

The more cycles I donate to others, the more the system as a whole would give back to me. This would be the bittorrent model for cpu sharing. Finding other nodes and distributing load could be done with many of the same algorithms that run TOR.