The Designer’s Task

I usually really enjoy having to design a computer application. Lately I’ve been trying to design a multithreaded server that’ll process image frames from a video. Right, that’s MPI, Network and video decoding all rolled into one. Decoding video into an event-driven stream of frames is difficult enough by itself, without having to worry about threads and network driven state changes simultaneously.

The task of the designer is to take complicated processes like the above, and distill them into a set of pieces that integrate together. The programmer will ususally loathe or relish the design process. It’s the point during which all the creative juices that brew during years of drudgery and long hours of typing finally get to surface. On a good day, the designer will get to wield Occam’s Razor like a sleek katana as they nimbly cut through all the conceptual cruft and end up with an elegantly trimmed and simplified piece of core functionality that not only “Just Works” but that’s easy for others to use, and provides aesthetic appeal. On a bad day, the cruftsman grafts parts together in ad-hoc fashion with duct-tape and a club until something works by sheer accident, some hideous kluge that deftly defies programmer and user expectations at every turn, dooming the sanity of everyone involved thereafter. Mostly the poor designs result from the designer not having the singularly clear vision of a guru, from not having spent enough time in deep meditations about the subtleties involved, from neglecting to see the ephemeral thread of commonality that hides neatly behind each of the disparate pieces being conjoined.

Unfortunately for me, I know next to nothing, and have no real experience with everything needed in this project. I’m not familiar with network programming or protocol design and implementation; I’m not familiar with event-driven architecture; I’ve only just figured out how to decode the video; and I’m only just now turning my attention to threads. Concurrency without Idempotency is Hard. The other real difficulty is that when I’m given the choice of where to allocate processing resources (via threads and MPI) there are several different, competeting ways to design the application. I’d much rather just code up a series of small idempotent functions and offload the difficulties of resource allocation to a well-designed multi-purpose scheduler.

As I think through the different designs, and read up on all of the skills I lack, I think it’s inevitable that I’ll design a kluge rather than a piece of art. I keep having moments of lucidity where I think I’ve got an elegant design, only to find later that there’s a gaping chasm surrounding a detail I overlooked in my ignorance. *sigh* I should probably resign myself to writing an inelegant hack, and plan on re-coding the whole thing, chalking up the initial attempt as a learning experience.