I'm privileged to have been invited to give a keynote session at the Modeling Wizards masterclass in Oslo, Sept. 30 -- Oct. 2. As you can see from the pictures, there's an impressive line-up of speakers: Jean Bézivin, Krzysztof Czarnecki, Øystein Haugen and other luminaries from the field of model-driven development.

Unlike other conferences and workshops, this isn't just people submitting their own papers. As the title and line-up maybe reveal, the idea is to offer the best possible training in MDD for the participants. The three-day program offers "a set of carefully selected lectures dealing with various aspects of modeling and with a particular focus on domain-specific languages. The objective is to provide each attendee with sufficient information to understand the main issues and challenges in the use of modeling and domain-specific languages, and also to have a clear picture of the most recent advances in the field."

One thing I'm particularly happy to see is a mini-project running each afternoon, where participants will get the chance to put what they are learning into practice, with a helping hand from the speakers. I'm a big believer in the master craftsman - apprentice mode of learning, and have benefitted greatly from it myself over the years. To ensure the personal attention necessary, places are limited -- so sign up now! The price of around 887€ includes accommodation and all meals for the three days, which compares very favorably with any other training I've seen. With the timing just before MODELS 2010, you can even get two events for the price of one set of air fares!

At this year's Code Generation conference in Cambridge, we're delighted to be able to offer our hands-on session teaching Domain-Specific Modeling with MetaEdit+, which was voted best session at last year's conference. We'll build five modeling languages and generators from scratch in 2%BD hours on Wednesday. If you've never created your own modeling language, or have only used things like GMF and think all tools must be similar, you have to see this!

That's a hands-on talk, but there are also wider issues: Most MDD projects fail before they even get started, or drag on but never really fulfill their promise. The reasons are as often human as technical. By knowing what tactics actually work and what don't -- many counter-intuitive or never considered by developers -- you can avoid the pitfalls and frustration of seeing your good ideas wasted.

We'll be looking at this in a new talk at Code Generation 2010: Proven best practices for successful MDD adoption (Thursday 10:45-12:00). This is intended for lead developers, team leads, architects, CTOs, managers - and also anybody trying to help or encourage other companies to adopt MDD.

The presentation will show you the roadmap for successfully introducing MDD into your organization, on both technical and human levels. Technical phases include the domain selection criteria, technical preparedness, language definition and testing, and the final roll-out. On the human level we will look at selling the idea, building mindshare, keeping momentum going, and managing organizational change. Drawing from our experience over the past fifteen years we will pinpoint the key challenges organizations usually face during MDD adoption, and offer some practical solutions to overcome them.

We will lead people through the technical tasks, people issues and process best practices in a tutorial format: teaching supported by slides, examples, cases, discussions and questions. Participants' own experiences and situations will be discussed throughout, so if you have a story to tell, do come along. We'll see if we can do a quick survey of participants to find which obstacles were most frequently encountered, and which were most serious.

Long time no blog. Aside from working on the next version of MetaEdit+ (about which more in a later post), I've been speaking at a variety of events. The first, back in November, was for the members of the International Association of Architects: a webcast on Domain-Specific Modeling: MDD that works. (The link will take you to the webcast, which was recorded live. Update: Added the codec and a Flash video alternative.) The format worked surprisingly well, even for the question and answer session: the participants could submit questions via the webcast chat tool, and Juha-Pekka picked a selection of them to ask me at the end. Something that was asked later was the size of the domain: with DSM, this is never something as broad as "embedded systems", or even "mobile phones". Normally it will be much narrower, e.g. a particular family or product line of mobile phones from a particular manufacturer, or then home and auto insurance policies in the EU. Making things no broader than necessary is one of the keys to high productivity with your DSM language, and also to making it easier to build the language: stick to what you are experts in, and follow YAGNI to avoid creating extra work for yourself.

One thing that struck me when preparing for the talk was that we often show graphs of how productivity increased in various DSM cases, but we don't normally have time to provide the details of each case. The cases studies are, however, available (see table below): some even with the modeling languages and generators publicly available. Most importantly, some of the cases obtained their productivity figures with scientifically rigorous experiments; not many companies have time for that, but those experiments serve to confirm the validity of engineers' estimates in other cases.

Productivity figures can seem a little abstract, so in the talk I looked at the in terms of something more concrete. Imagine that you can build one product in a certain time by coding. How much more can you build if you apply various kinds of modeling? In a previous post we already saw how with normal UML, you can actually build less than with manual coding. With MDA, the tool vendors' own figures range from 22% extra (Obeo: average cost reduction of an MDA project is 18% => 22% productivity increase) to 35% extra (OptimalJ: 30%201340%). You can see from the red bars on MDA1 and MDA2 below just how little you get for your investment in tools and training %2014 not to mention the long term nightmare of trying to maintain CIM, PIM, PSM and code in synch with only semi-automatic support. The productivity results for DSM tell a very different story: 5%201310x faster, and completely automatic code generation.

But what is the cost of creating your own DSM language and generator? After all, the MDA tool vendors promise to provide those, so that's an extra cost for DSM. Here are the figures in person days for the cases above. I think they speak for themselves: 2%20133 weeks by one person is so small it makes you cry. How many companies out there could spend 2%20133 weeks to make themselves 5%201310x faster?

Of course those figures demand good tools: all those cases are with MetaEdit+. Your mileage will definitely vary with other tools%A0%2014 and what's worse, because they make you do more work, your brain power will be directed away from creating the best possible language. At the end of the day, it's the size of the productivity improvement for the modelers that counts the most. If you can make 10 modelers 6x faster rather than 5x faster, that's worth 10 extra person years per year. With MetaEdit+, you're far more likely to get those extra person years, and you're sure to get your language ready faster. When we see the first case with GMF or DSL Tools making it into the 5 10x range, we can look at this topic again...

Referenced DSM Cases (for more see www.dsmforum.org)

In the keynote at Code Generation, I mentioned that empirical research shows that using UML does not improve software development productivity: depending on the study, reports ranged from -15% to +10% compared to just coding. I guess most people these days know those results from their own experience, but as the reports I was aware of were from the 1990s, it was interesting to see a more up-to-date article recently:

WJ Dzidek, E Arisholm, LC Briand: A Realistic Empirical Evaluation of the Costs and Benefits of UML in Software Maintenance, IEEE Transactions on Software Engineering, Vol 34 No 3, May/June 2008

Unlike many earlier studies, this uses professional developers and reasonably large tasks. The tasks all extended the same Java Struts web application, in total about 30 hours per developer. 10 developers performed the tasks with Java, and another 10 performed the same tasks with Java and Borland's Together UML tool. The developers using UML were somewhat more experienced -- 256 kLOC of Java under their belts rather than 187 kLOC, and 44% longer Struts experience -- but otherwise the groups were similar. Time was measured until submission of a correct solution, giving a reasonably sound basis for comparison. Here are the results:

Compared to just coding, using UML took 15% longer to reach a correct solution (the green bar). In addition, it looks like even using UML to help you understand the code gives no benefit over just reading the code: the blue and red bars are the same length as the purple bar. As the tasks only looked at extending an existing system with existing models, we can't say for sure whether the story is the same in initial implementation, but other studies indicate it.

One bad thing about the article is that it tries to obfuscate this clear result by subtracting the time spent on updating the models: the whole times are there, but the abstract, intro and conclusions concentrate on the doctored numbers, trying to show that UML is no slower. Worse, the authors try to give the impression that the results without UML contained more errors -- although they clearly state that they measured the time to a correct submission. They claim a "54% increase in functional correctness", which sounded impressive. However, alarm bells started ringing when I saw the actual data even shows a 100% increase in correctness for one task. That would mean all the UML solutions were totally correct, and all the non-UML solutions were totally wrong, wouldn't it? But not in their world: what it actually meant was that out of 10 non-UML developers, all their submissions were correct apart from one mistake made by one developer in an early submission, but which he later corrected. Since none of the UML developers made a mistake in their initial submissions of that particular task, they calculated a 100% difference, and try to claim that as a 100% improvement in correctness -- ludicrous!

To calculate correctness they should really have had a number of things that had to be correct, e.g. 20 function points. Calculated like that, the value for 1 mistake would drop by a factor of 20, down from 100% to just 5% for that developer, and 0.5% over all non-UML developers. I'm pretty sure that calculated like that there would be no statistically significant difference left. Even if there was, times were measured until all mistakes were corrected, so all it would mean is that the non-UML developers were more likely to submit a code change for testing before it was completely correct. Quite possibly the extra 15% of time spent on updating the models gave the developer time to notice a mistake, perhaps when updating that part of the model, and so he went straight back to making a fix rather than first submitting his code for testing. In any case, to reach the same eventual level of quality took 15% longer with UML than without: if you have a quality standard to meet, using UML won't make you get there any more certainly, it will just slow you down.

To their credit, the authors point out two similar experiments as related work. One showed UML took 27% longer, the other 48% longer. The percentage of time spent updating models was also larger: 30-35% (which may be because those studies only measured time until the first submission of a solution: correcting bugs was probably mostly coding, so if measured to a correct solution the UML time would only increase a little and hence the percentages would drop).

So what do we learn from all this? Probably nothing new about UML, but at least a confirmation that earlier results still apply, even for real developers on realistic projects using today's UML tools. Maybe more importantly, we can see that empirical research, properly written up, is valuable in helping us decide whether something really improves productivity or not. Ignore the conclusions (they probably existed in the minds of the authors before the paper was written), but look at the data and the analysis. Throw out the chaff, and draw your own conclusions from what is left. Above all, don't blindly accept or reject what they say, just because it agrees or disagrees with your existing prejudice. There's at least a chance that you might learn something!

Once again, Code Generation proved itself as the best European conference on Model-Driven Development. Lots of smart people, lots of experience, lots of enthusiasm, lots of willingness to listen and learn from others. Even though having to prepare and run some sessions hampered me from seeing as much of the rest as I'd like, there's still too much to write for one blog post. I'll post about things I'm certain of first, and come back to things like Xtext and MPS after further investigation.

Keynotes

The two keynotes, both presented as a double act by me and Markus Völter, seemed to go down well. Mark Dalgarno had a surprise up his sleeve, presenting us with a blind choice of weapons from a black bag. We then had to duel it out, graphical DSM against textual DSLs, with the plastic gun and dagger we picked. Since I got the gun, I think the result was a foregone conclusion :-). The dagger may be a "weapon from an earlier, more civilized age", but it's only useful if you can get in close to your adversary. Similarly, text may be more familiar, but it does often tie you closer to the code; problem domain DSLs in text seem as rare as accurate knife throwers. Markus successfully stabbed me in the back later on, so that evened things up and emphasized the point from our slides: both text and graphics are useful in the right place. Choose, but choose wisely.

It was fun to see the keynote get picked up on Twitter:

EelcoVisser: keynote by @markusvoelter and Steven Kelly at #cg2009: great overview of issues in model-driven development

HBehrens: Steven Kelly at #cg2009 keynote: "wizard based generators create a large legacy application you've never seen before"

The latter was picked up by several people. The reference was to vendor-supplied wizards, often found in IDEs or SDKs, that create skeleton applications for you based on your input. Since the vendors take pride in just how much boilerplate they can spew out, you're left with a mass of generated code that you've never seen before, but must extend with your own code. Worse, you're responsible for maintaining the whole ensuing mixture, and there's no chance of re-running the wizard to change some of the choices -- at least not without losing or invalidating the code you've added. That's in sharp contrast with generation in DSM, where your input is in the form of a model which you can edit at any time. You get the speed of generation but can remain at a high level of abstraction throughout.

MetaEdit+ Hands-on

We'd decided to try something special in the hands-on: building 5 different graphical modeling languages from scratch in under 3 hours. Rather than being random exercises, the languages were increasingly good ways of modeling the same domain. We started with something that was basically just the current code turned into graphics, and ended up with a language that reduced the modeling work to a third of what it was at its worst, with many possible errors ruled out by the language design and rules, and with much better scope for reuse. We showed how to make generators for all the languages, and actually built them for two. And of course since this was MetaEdit+, simply defining the metamodel already gave you a full graphical modeling environment -- we just tweaked the symbols to taste.

Never having run the session before, we were rather nervous about how much we could achieve in the time available. In the end, thanks to great slides from Risto Pohjonen and testing from Janne Luoma, it seems we pretty much hit our target. Only at the very end of the last language did we have some people only just starting the last section (the generator) while others were finishing it and going on to beautify the symbols or play around with other fun features of MetaEdit+. Hopefully people learned not just about MetaEdit+ as a tool, but also how to make better languages and improve existing ones. Feedback online was encouraging:

PeterBell: Great metaedit hands on - built and refactored language and generator in just a couple of hours at #cg2009

elsvene: been to a great hands-on session for MetaEdit+. Really interesting tool! #cg2009

HBehrens: for me MetaEdit is the most sophisticated graphical modeling tool currently available #cg2009. Thanks for this session!

Dinner

The conference dinner was of the high standard you'd expect from a Cambridge college. The airy hall and contemporary art lent a friendly ambience. The large round tables weren't particularly conducive to conversation: you could only really talk to the people either side of you without shouting or craning your neck. On long tables you can reach 5 people for the same effort. I was fortunate to be sitting between Scott Finnie and Jon Hurwitz, so I certainly didn't suffer.

The "suffering" started later, when there was a raffle in aid of Bletchley Park, the home of Allied code-breaking work in World War II. I ended up winning a prize donated by Microsoft: a screwdriver toolkit and MSDN T-shirt, causing much hilarity and bad jokes about finally getting Microsoft tools that didn't crash. The irony continued when Alan Cameron Wills won a signed copy of our Domain-Specific Modeling book -- despite having received one from us last year. Either the older British segment of the audience were most inclined to support Bletchley Park by buying raffle tickets, or then the draw was rigged to encourage vendor co-operation. The people on my table were having none of that, and encouraged me to cover up the Microsoft logos :-). All in all a good laugh, and in a good cause.

Markus Voelter and I are having fun at the moment preparing our keynotes for Code Generation. The descriptions on the web page are deliberately vague, but the important fact is there: we'll be giving both keynotes together.

As frequent conference attendees will know, Markus and I are both quiet, meek guys who would never presume to disagree, so the talks will most likely be boring consensus... NOT! I did suggest mud wrestling would be an easier way to settle our differences, but my imposing physical presence must have convinced Markus he'd have a better chance with PowerPoints at twenty paces.

In related news, Mark Dalgarno has finally realized that the concepts of "early bird" and "software developer" make uneasy bed-fellows, and the way to get people to sign up some reasonable time before conferences is to use the stick not the carrot. Yes, there's now a special not-very-early-bird price increase of 10% extra heading your way if you don't go to the site NOW and register.

It's not all stick though: if you were there at either previous conference, you get 5% off. Canny forward thinker that he is, and with CodeGen 2026 clearly in mind, Mark isn't offering 10% off if you were there both years (darn!).

However you cut it, Code Generation is simply the best conference on DSM in Europe. Even without the mud wrestling.

The roadmap for Martin Fowler's forthcoming book on DSLs indicates that he will focus on textual DSLs. The online draft of the intro does however briefly show a graphical language for a home security system: the model in Figure 6 is implemented with MetaEdit+, based on the original textual requirements:

Miss Grant has a secret compartment in her bedroom that is normally locked and concealed. To open it she has to close the door, open the second draw in her chest, turn her bedside light on - and then the secret panel is unlocked for her to open.

Juha-Pekka has been using Martin's example as a way of showing how to implement a DSM language in MetaEdit+ (Parts 1 and 2 ), and in Part 3 he points out some problems with the original language: too broad a focus, unclear usage process, and too low a level of abstraction. Juha-Pekka correctly suggests going back to the basics of the domain to discover the necessary language concepts, rather than trying to shoehorn this domain into a generic state model.

As an exercise, however, I thought it might be interesting to try to improve Martin's language as it is, rather than starting from scratch. How much of DSM is "you just have to know how to do it", and how much can be reduced to simple steps that anyone could apply? Obviously, the more of the latter that we can find, the easier it is for somebody to get started. Our DSM book aimed at just this kind of practical approach; let's take a few hints from there and apply them to Martin's language. We'll show the model in the current state of the language as we evolve it: click the pictures to see the full size screenshot.

Use meaningful symbols

Martin's language uses just black and white shapes, the kind you might see in a standard flow chart palette. Only the text within the shapes gives a clue as to the actual domain: words like "door", "drawer", "light" and "panel" occur many times. However, the brain takes a lot longer to find all occurrences of a word in a picture than it does to find all occurrences of a symbol. Try it yourself: how many times does the door symbol appear in the picture on the right, and how many times does the word "door" appear in Martin's Figure 6? (You'll actually notice a slight discrepancy: Martin's diagram omits the "reset all / return to start" event caused by the door opening, shown at the extra door at the top left in our diagram; he mentions this elsewhere in the draft.)

Reuse objects

Having four door objects like this obviously isn't ideal: Martin had them, but the problem wasn't so visible there because they a) couldn't be distinguished from other objects, and b) didn't so clearly represent something in the physical world -- the problem domain. Now that we have them visible, it would be nicer if we could show that there's really only one door in this model, and it is involved in four different events or actions. So let's merge the four doors into one, and similarly for the panel.

The light bulbs and drawers are harder: if we merge them, we end up with either lots of crossing lines, or objects on top of each other -- ugly. Maybe there's something else we could do for them?

Consider n-ary relationships

N-ary relationships -- relationships involving more than two objects -- are everywhere: a "family" relationship links a father, mother and children; an inheritance relationship links a superclass with several subclasses. When people draw a diagram on paper, they're happy drawing lines that split. However, implementers of modeling tools have often misanalyzed the simplest and most common case of a binary relationship, and ended up thinking relationships can only connect two objects. They end up having to represent n-ary relationships with a fake object in place of the relationship. Such fake relationship objects leave the modeling language inconsistent, as the user can draw a "relationship object" on its own without connecting it to anything. They also make checking model correctness much harder, as the rule for what can be connected in a certain kind of relationship must be split over several relationships, all cobbled back together through the fake relationship object. (For more details, see Welke's article from my previous entry on The Model Repository.)

If you're lucky enough to have a tool that supports n-ary relationships properly, take a look at your modeling language and see if you can make a more complex structure of objects into a simpler one by connecting several objects with a single relationship. In this case, Martin already shows excellent taste by using n-ary relationships for transitions :-) -- but maybe we can go a bit further still. On the left path between Active and Unlocked panel we can see the sequence "Drawer opens", "Waiting for light", "Light on"; on the right path we have "Light on", "Waiting for drawer", "Drawer opens". If we go back to the original text, we can see that all this means is that we wait for the drawer to open and the light to come on: both must happen, but in either order. So why not just have a single transition with two events to trigger it? We can make that the default semantics of a transition: it waits for all attached events to happen, in any order. To remind ourselves that such transitions will wait for all events, we show a little block on them where the lines meet. If we wanted to support the case where either one event or the other could happen, but not both (XOR), we could have a property in the transition to specify whether it is AND or XOR, or then simply require the user to draw two transitions between the two states. In either case the amount of extra work for the XOR case is much less than is needed in a generic state model for the AND case, which requires the insertion of extra "Waiting for..." states -- only two here, but imagine covering all possibilities if there were 5 events that could happen in any order. (Exercise for the reader: how many "Waiting for" states would that require? Hint: we can do better than 5 factorial.)

Rule out corner cases

An important feature of good DSM languages is that they make the job of the modeler easier. In Martin's language, one of the hard things to see is in which states are the panels, doors etc. unlocked. We can see the actions that unlock them, but to know the state of a panel in a given state, we need to play through all possible routes that can get to that state. As the whole point of this language is to describe when things are locked or unlocked, this is quite a serious problem. Is there a way that we can make things clearer to the modeler? If we look at a few of these models, we see a common pattern emerge. On the right here a panel is unlocked by a state "Panel unlocked", and there is a transition from that state when the panel is closed, to a state that locks the panel again. This unlock->close->lock sequence appears in many models, and makes sense in the problem domain. So why not allow a shortcut syntax, where the panel itself plays the role of a state, as at the bottom in this picture: on entering the panel state, the panel is unlocked; we leave the panel state when the panel is closed, and on leaving it we lock the panel. Since we can specify the semantics like this, we can obviously make the generators produce the required code: we can do this by extra steps in the generator, by a model-to-model transformation that produces a more generic state machine, or by a more powerful state machine engine in the framework. The last would be my choice, as that way the code generated per model has the closest resemblance to the models, stays on a higher level of abstraction, and keeps the overall size of the application down.

That takes care of the case where the panel unlock-close-lock sequence can be considered as an atomic element in the model, with nothing else happening during it. What about cases like the door being unlocked, during which there is a sequence of other events needed before it is again locked? In this case we can use a sub-model: in the figure above, the green padlock relationship connecting the "Door unlocked" state to the door means "during this state, the door is unlocked" -- i.e. on entry the door is unlocked, and on exit it is locked; as before, closing the door exits that state. The little blue star in the "Door unlocked" state indicates that it has a submodel, shown in this figure. The contents of the submodel are of course just the set of states during which the door is always unlocked. Now it's easy for the modeler to know whether the various secret compartments are locked or unlocked at each stage -- and of course thus to ensure that the system he designs has no holes in its security. And since the code is generated, we'll never forget those pesky bounds checks, so there'll be no buffer overruns to exploit :-).

Keep models compact

Sub-models are great for hiding complexity and making the modeling language scale better. If each model becomes too small, however, many people find it harder to understand. Those with a Lisp or Smalltalk background are used to methods being only 2-4 lines of code; in more commonly used languages several such methods tend to be grouped together into a single larger method. Obviously extremes in either direction are bad; providing we stay within the bounds of what is sensible, we can choose the option that the modeler feels more comfortable with. In this diagram we have combined the two small models back into one larger one, stretching the "Door unlocked" state to enclose its substates. We can still see during which states the door and panel are unlocked, and maybe the overall picture is clearer -- or maybe not. In any case, this slightly reduces the number of model elements compared to the previous step.

Metrics

If we count each object as 1 element, each binary relationship as 1 element, and each additional role or property as 1 element, plus 1 for each model, we get the following size metrics for the models above:

As can be seen, we've reduced the size of the model by over 50%. Since the effort needed for a given project increases more than linearly with size, we can estimate that productivity increases compared to the original language by a factor of at least 2. Improving symbols and cutting out corner cases weren't aimed at reducing the size of the model, but will have significant improvements in usability, so I'd guess overall a factor of around 3 is reasonable. Note that this is on top of whatever improvement is gained by Martin in moving from a straight hand-coding solution to a DSL, and from a textual DSL to a graphical DSL. More importantly, though, these are the kinds of steps that anyone can see how to apply to their own modeling language, and any team of developers would benefit from at the modeling level. Interestingly, with MetaEdit+ all of these changes could be applied to the modeling language without throwing away the initial model: the initial model and all intermediate models remain valid throughout, as the language evolves.