Measuring Effectiveness of a Domain Specific Language

Also, at CGO I met Hassan Chafi, who is working on a graph-based Domain Specific Language. Even though I never seem to find time that I can explicitly devote to studying them, DSL’s are, to me, an compulsively fascinating topic. A day or so after the discussion it occurred to me that we need some metrics by which a DSL can be measured. Now, in the general purpose language field Wirth has come up with what is, in my opinion, a very elegant metric: language complexity can be measured by the size of the self-hosting compiler. That works great for general purpose languages that have to do string processing, parsing, data structures, traversals, modeling, etc. Each of which is a component of the self-hosting compiler. But it works less well for a DSL, because the focus on particular domain means they aren’t general purpose.

In the case of a DSL for graphs though, I think the case is clear: It should run graph algorithms well. But which ones? And how do you measure expressibility? It took a couple of days for the answer to arrive in my head. I had at one point encountered a wonderful paper on On variants of shortest-path betweenness centrality and their generic computation by Ulrik Brandes. This paper provides a dozen related graph algorithms. It is presented in a way that emphasizes the changes between the base centrality algorithm and each variant. This style of presentation helps to measure how well the DSL allows similar algorithmic changes.

So I think it’s a good start to answer the general question, “How to measure the effectiveness of a DSL?”, with a case study. Make a list representative of what you wish to do, and try it out, looking for patterns and variations on a theme.