Friday, December 6, 2013

Where we stand in the world

Here is a map of the world depicting the 15 non-EU members of G20 (red) and the 28 EU states (green).

G20 and EU
According to the site, the G20 represents about 85% of global GDP, over 75% of global trade and 66% of the world's population. Confronting this with a Wikipedia page on G20, one finds out that the percentage for the global trade is slightly off, because it includes Germany, France, UK and Italy numbers twice - once for these countries themselves and once for their EU membership. Thus, the correct number seems to be rather 58% of global trade for G20.

Now, if we take the EU completely out, just to see against what economic and geopolitical power Europe competes, we get something which could be called a G15 (G15 = G20 - Germany - France - UK - Italy - EU). Note that there seems to be an informal group of countries of the same name, so do not confuse it with our G15.

G15, even after the removal of the entire EU, is still a major group, representing 63% of global GDP, 45% of global trade and 57% of world's population. As a side note, the majority (8) of these countries are federal states (also large and multiethnic) or at least declare themselves as such. The smallest G15 member, Australia has population of only 22 million. The middle-sized member, Mexico, has 112 million inhabitants and the two largest countries in this group, China and India, have 1.2 and 1.3 billion people living in them, respectively.

When we put the EU into the same perspective (these are all figures from 2012, so Croatia is not included, sorry), we find that it is responsible for 23% of global GDP, 12% of global world trade and has 7% of the world's population. Note that this is not even the whole of Europe, just EU-27.

Let us make one additional daring step and have a look at how e.g. UK, a country that sometimes likes to picture itself as an independent state outside of the EU, performs in this global game. UK, as an independent country, creates around 3.4% of global GDP, 3% of global world trade and has about 0.89% of the world's population. Just draw your own conclusion, considering also the dynamics of the UK and G15 economies and populations.

After having opened this blog entry with a picture, would there be a better way of concluding it other than using another picture to illustrate the dramatic change through which Europe's position in the world has gone since 1945? Here is a map showing, among other things, colonies of several European countries in 1945. Colonies that are no longer there and will never come back.

Colonialism in 1945, source: Wikipedia

G20 members represent around 85 per cent of global gross domestic product, over 75 per cent of global trade, and two-thirds of the world’s population. - See more at:
G20 members represent around 85 per cent of global gross domestic product, over 75 per cent of global trade, and two-thirds of the world’s population. - See more at:
G20 members represent around 85 per cent of global gross domestic product, over 75 per cent of global trade, and two-thirds of the world’s population. - See more at:
G20 members represent around 85 per cent of global gross domestic product, over 75 per cent of global trade, and two-thirds of the world’s population. - See more at:
G20 members represent around 85 per cent of global gross domestic product, over 75 per cent of global trade, and two-thirds of the world’s population. - See more at:
G20 members represent around 85 per cent of global gross domestic product, over 75 per cent of global trade, and two-thirds of the world’s population. - See more at:
G20 members represent around 85 per cent of global gross domestic product, over 75 per cent of global trade, and two-thirds of the world’s population. - See more at:

Sunday, June 23, 2013

Connecting two HelenOS guests in QEMU using SLIP

I've been recently working on implementing the SLIP protocol support for HelenOS, represented by the following ticket:

#439 SLIP IP link provider
It turned out that it was actually much more difficult to setup the environment for QEMU than to implement the actual SLIP support, even though on the HelenOS front I had to work around a known HelenOS IPC deficiency:
#508 Parallel sessions don't mix well with call forwarding
The basic SLIP support has been integrated into HelenOS in mainline revision 1849.

The following describes the steps that one needs to make in order to have two HelenOS instances running in QEMU talk to each other using SLIP. First of all, make sure to have a recent version of QEMU installed as versions prior to QEMU 1.5.0 may hang when dealing with the redirected serial port.

Since we are going to redirect and connect the serial ports of both HelenOS guests via two named pipes, we need to make some preparations first. On the host system, replicate the following steps:

$ mkfifo /tmp/ /tmp/fifoA.out
$ ln -s /tmp/fifoA.out /tmp/
$ ln -s /tmp/ /tmp/fifoB.out

Now, launch two HelenOS guests, each with its own pipe. Note that the symbolic links representing the fifo pair B are intentionally crossed so that QEMU opens each pipe in a proper way:

$ qemu-system-i386 -cdrom image.iso -serial pipe:/tmp/fifoA &
$ qemu-system-i386 -cdrom image.iso -serial pipe:/tmp/fifoB &

In the first HelenOS instance, run the following commands:

/ # slip devices/\hw\pci0\00:01.0\com1\a net/sl0
/ # inet create net/sl0 ma
/ # inet add-sr mr

And in the other HelenOS instance, run this set of commands:

/ # slip devices/\hw\pci0\00:01.0\com1\a net/sl0
/ # inet create net/sl0 ma
/ # inet add-sr mr

At this point, you should be able to ping the first instance from the second one and vice versa:

Friday, May 10, 2013

Hitting the European nail on its head

I've been considering writing up about what I have been recently up to with these European Federalist Papers for a longer time, but this Friday, a day after this year's Europe Day, is the first convenient moment. It is no secret that I am a great supporter of the concept of strong and united Europe, but no, I am not entering politics or spreading the official Brussels "propaganda" as some of you might be wrongly suspecting. As you will see, it's something rather different.

Instead, I became interested in a private undertaking of three European citizens - experts in the fields of law and administration, and federalists at the same time - who decided to try to do something about the current state of things in Europe. Before I write more about what these three men did, let me make a short historical stop in North America at the times of the Philadelphia Convention, i.e. the end of the 18th century.

In 1787, when there was a danger of State of New York rejecting the new federal Constitution of the United States, three other men - Alexander Hamilton, James Madison and John Jay, all Americans and federalists for that matter - wrote and published 85 newspaper essays explaining and promoting the new constitution. The high quality and a tremendous pace of their work helped to reverse the negative development in favor of New York's ratification of the United States Constitution.

The essays - collectively known as The Federalist and later also as the Federalist Papers - have become a kind of a legend of their own. They still are, more than 220 years after being written, used by the US authorities to reason about the meaning of the US Constitution and the intentions of the America's Founding Fathers.  When I read the first dozen of them during the last summer, I had wished something like that had existed for Europe.

At that time I could not have known that, alarmed by the worsening Euro crisis, Leo Klinkers, Herbert Tombeur and Fernand Jadoul already started writing their European Federalist Papers. Inspired by the American version and knowing that the skeptics all around Europe have become very busy advocating for disunion, I was even considering and wanting to write something myself, but something much better and more fortunate happened: on January 14 of this year I randomly ran across a tweet announcing the series of 26 European Federalist Papers. As a reader, I have been following the Papers ever since and what I learned from them made me step out from the passive role of a mere reader. Read on to know what exactly I did, but first, let me say what I find so great about these Papers.

First, the Papers make a direct comparison between the USA under the Articles of Confederation and the EU under the Treaty of Lisbon. I was previously laughed at by the illiterate when I tried to compare these two systems and draw any conclusion from the striking similarity of their respective crises and ways out of them. So starting from this position immediately gained the Papers my sympathies.

Second, the Papers are a real eye-opener. I must admit that before reading the Papers I probably could not see that federal Europe cannot be achieved by adjusting the intergovernmental treaty, from the will of the heads of governments. Which is worse, I did not understand what a federal Europe means. I had just a rough idea of European mutuality, but didn't really make any difference between a genuine constitution, an intergovernmental treaty or a constitutional treaty. My thinking in this area was locked inside of a box.

Third, the Papers show no restraints to criticize the current malfunctioning intergovernmental system of the European Union, personified in the European Council, and don't go far to denounce the Treaty of Lisbon. This makes the Papers totally independent and no-one can say this is some paid propaganda from Berlaymont. The papers are concluded by a draft European Constitution. The constitution starts with the words "We the Citizens" and has only ten articles that fit on a couple of pages, strongly contrasting with the 400-page-or-so Treaty of Lisbon.

Lastly, the European Papers are trying to follow the American best practice as closely as possible. This includes the emphasis on the tremendous citizens` role in the process of forming the federation, in a bottom-up manner. This effort is also apparent from the fact that the European Federalist Papers are not an official EU memo, they do not represent any Member State's position, they were not created by any unelected officer in Brussels, no head of government is responsible for them. Instead, it is pure citizenship and passion for the idea of federal Europe which lead to the creation of these Papers.

This citizens` dimension - everything comes from the citizens: the European Federalist Papers themselves, the Draft Constitution and the prospective convent and ratification reminds me the workings of an open source software community. And that's the reason I wanted to start contributing myself, to become one of the active community members.

I realized that the Papers, until then available only in English and Dutch, do not reach out to all the potentially interested people as there are many people who do not speak any of these two languages. So I offered my help and to this date, I translated Papers 0 - 3 into Czech. I also encouraged and organized others to translate the Papers into their own language or to help the existing translators to share the massive burden of translating by parallelizing their efforts. So far, there should be several other translations in progress.

At the same time, the authors of the Papers realized the power of the modern social media and instead of choosing the classical newspapers, they chose Web and Twitter as means to spread the federalist message. This was, of course, an idea that I appreciated too, even though it was clear that its implementation can be improved. Using my experience with community building, I started European Federalist Papers pages on Facebook and Google+. And that's probably how you learned about my involvement with the European Federalist Papers.

Now, my countrymen, when you know what I have been up to, you may as well consider supporting this noble idea of true federal Europe. If nothing else, I kindly ask you to read the papers as that's where everything begins. The rest is up to you.

Monday, February 11, 2013

Horizontal or vertical scalability?

Just finished reading a networking paper originating in the MINIX group called Keep Net Working - On a Dependable and Fast Networking Stack. Before going into the real meat of the paper, I must first grumble a little because the paper completely withholds the fact that HelenOS was the first multiserver microkernel-based system to have a fully decomposed networking stack, as described in Lukáš Mejdrech's 2009 master thesis Networking and TCP/IP stack for HelenOS system and briefed about by me at FOSDEM 2012. True, the first HelenOS networking stack was not a big success and we have reimplemented most of it since then, but HelenOS still has a fully decomposed networking stack. I suspect the authors of the paper must have suffered with temporary amnesia when writing the following words in 2012:
By chopping up the networking stack into many more components than in any other system we know...
Putting an end to my grumbling, the HelenOS networking stack has not been decomposed for the sake of saturating multi-gigabit network interfaces nor for self-healing capabilities, which are the areas the paper indeed attempts to innovate in.

What is more interesting about the paper is that it assumes processor cores are becoming an increasingly cheap resource and suggests to dedicate individual cores to networking stack components. So far so good and it is likely that this approach greatly improves the performance of the completely non-scalable legacy MINIX 3 networking stack. The problem, as I see it, is that this improvement seems to have gone into a dead end and there is hardly a way for it to keep the same design while still bringing new performance gains at the same time. The reason I think this is actually the case is the evolution of the Solaris 10 networking stack as described in the FireEngine whitepaper. According to both the MINIX group paper and the Solaris whitepaper, the two have gone into completely opposite directions.

The Solaris networking stack has abandoned the idea of horizontal networking stack scalability in which every component is serviced by a couple of kernel threads that can all run on a different processor in favor of a model in which the networking stack is multithreaded and each processor has de facto its own local instance of it. This vertical setup allows for a packet to be processed entirely on one processor, greatly improving cache utilization. Of course, this approach scales with the number of processors because the Solaris stack can be processing as many packets in parallel as there are CPUs without suffering from poor data locality.

On the other hand, the MINIX group paper suggest to have each networking stack component wired to a dedicated processor, so that each packet travels horizontally across the several processors as it is being processed by the networking stack. The paper praises this design's cache utilization because it appears the stack components can run without kernel intervention most of the time so that the caches remain populated with the component's code and data, but it completely ignores the effects caused by the packet itself changing processors and caches constantly.

It may be that the costs and benefits of each approach are different for a monolithic kernel such as Solaris and a multiserver MINIX 3 derivative such as the one described in the paper, but one thing is for sure. The lessons learned during the evolution of the Solaris networking stack warrant a very interesting comparison study.

Friday, December 21, 2012

Lisbon Treaty voting system: WIN^27 situation

I have recently become puzzled by a simple question. Does the Lisbon Treaty actually weaken the position of the Czech Republic in the Council of the European Union compared to the Treaty of Nice? I therefore went ahead and studied both, actually all three, basic variants of the voting system: the plain Nice voting system, the extended Nice voting system when a member state wants to bring the population share criterion in, and the Lisbon voting system. Besides some basic observations, I could not come to a general conclusion as I was able to find scenarios in which a member state both won and lost, depending on the treaty in use.

I clearly needed to consider all possibilities and evaluate them to be able to come with a clear judgement. Given there are currently 27 member states, there are exactly 2^27, or 134217728, different votes in which all member states participate and vote either for or against the proposal. This is still a small enough problem space for the modern computer equipment to exhaustively explore in a reasonable amount of time so I went ahead and constructed a small Python script which goes through all possible votes and evaluates them with respect to all three considered voting systems and the interest of the state from whose perspective the evaluation is made. I soon realized I can compute the results for all member states, not only for Czech Republic. Here are the results.

Member state Votes (Nice) Votes share (Nice) Population share Wins (Nice) Wins (Nice+) Wins (Lisbon) Lisbon improvement
Austria 10 2,90% 1,67% 67980524 67980509 70037478 1,53%
Belgium 12 3,48% 2,17% 68147346 68147331 70380860 1,66%
Bulgaria 10 2,90% 1,49% 67980524 67980509 69914262 1,44%
Cyprus 4 1,16% 0,16% 67461254 67461241 68996390 1,14%
Czech Republic 12 3,48% 2,10% 68147346 68147331 70332762 1,63%
Denmark 7 2,03% 1,11% 67723582 67723567 69653426 1,44%
Estonia 4 1,16% 0,27% 67461254 67461241 69071824 1,20%
Finland 7 2,03% 1,70% 67723582 67723567 69625576 1,42%
France 29 8,41% 12,95% 69302512 69302519 77686128 6,25%
Germany 29 8,41% 16,27% 69302512 69302527 80252488 8,16%
Greece 12 3,48% 2,25% 68147346 68147331 70436030 1,71%
Hungary 12 3,48% 1,99% 68147346 68147331 70257314 1,57%
Ireland 7 2,03% 1,70% 67723582 67723567 69625852 1,42%
Italy 29 8,41% 12,70% 69302512 69302505 77037338 5,76%
Latvia 4 1,16% 0,44% 67461254 67461243 69189842 1,29%
Lithuania 7 2,03% 0,65% 67723582 67723567 69335640 1,20%
Luxembourg 4 1,16% 0,10% 67461254 67461241 68955580 1,11%
Malta 3 0,87% 0,08% 67374448 67374441 68939830 1,17%
Netherlands 13 3,77% 3,31% 68228986 68228971 71165244 2,19%
Poland 27 7,83% 7,60% 69200228 69200213 73490170 3,20%
Portugal 12 3,48% 2,12% 68147346 68147331 70345358 1,64%
Romania 14 4,06% 4,26% 68309352 68309337 71839368 2,63%
Slovakia 7 2,03% 1,80% 67723582 67723567 69631656 1,42%
Slovenia 4 1,16% 0,41% 67461254 67461243 69168020 1,27%
Spain 27 7,83% 9,18% 69200228 69200231 74883076 4,23%
Sweden 10 2,90% 1,87% 67980524 67980509 70173958 1,63%
United Kingdom 29 8,41% 12,43% 69302512 69302509 77295044 5,95%

Note that there are also votes in which not all member states participate due to an opt-out from some policy area or in which some member states abstain from the vote. I did not consider this in my simulation as the problem space would significantly increase in size to 3^27, or 7625597484987, and thus make the simulation unacceptably long. Moreover, the voting systems tend to get complex when this situation occurs. Also note that for the sake of simulation, my script assumed rules that apply for Commission proposals. Proposals not initiated by the Commission need to meet stronger criteria than ordinary proposals.

As for the results, the chart above clearly shows that all member states will benefit from the transition to the Lisbon Treaty voting system in 2014/2017 as they will win the vote in more cases. Winning the vote in this case means that the proposal is accepted when the state votes in favor of it and rejected when it votes against the proposal. The table does not indicate whether the won votes for the proposal prevail over the won votes against the proposal or vice versa. The table also shows that the benefit is smaller for smaller states and bigger for bigger states.

There is one small trouble with the result. Even though the Lisbon Treaty comes out victorious from this comparison, we don't know which are the important votes so even though all member states win the majority of votes, the really important ones may fall into the lost minority. To solve this problem, we would need to know in advance what are the essential proposals and how will the member states vote on each of them. The possession of such knowledge would, however, be equivalent to having a magical crystal ball. For the time being, the purely statistical outlook will need to suffice.

Tuesday, December 4, 2012

HelenOS and getting a job

It has become an increasingly common pattern that a recruiter from a global software company keeps track of people who had previously worked on and contributed to some HelenOS project, and tries to hire them. I think we can even go as far as saying that some recruiters have a special weakness for the HelenOS team. I have figured this out only recently from the various anecdotes people tell during the project meetings and, frankly, I am glad that work on HelenOS can be at the beginning of the process of getting a new job for somebody.

It doesn't take a recruiter who is already familiar with HelenOS and your work on it to get a job in our industry. You can use your HelenOS experience and track record from your own initiative. The purpose of this blog post is to encourage you - our contemporary and former contributors - to use your HelenOS achievements as a leverage during your interviews.

I have obviously done the same thing in the past during my own interviews. My overall impression then was that perhaps I had not been stressing the value of my HelenOS experience strongly enough or that the interviewer had tendencies to dismiss it as some sort of a school project. In a hindsight, I wish I had been advocating with greater confidence. Anyway, I have good reasons to believe that HelenOS has already helped me to get two nice jobs (yeah, the interviewers who didn't get it were from some of the companies for which I did not end up working, naturally).

Here are some things to consider when being in the position of the interviewee (and, as it reportedly already happened, also the interviewer):

  • HelenOS has long since outgrown the status of a school project; Ohloh estimates that it would take roughly 131 person-years to recreate HelenOS. Despite some slight duplicities in this figure resulting from the repository switch in the middle of 2009, and given that the vast majority of its code was written from scratch directly by us, this is a heck of an achievement. Do not hesitate to claim your part.
  • HelenOS is evolving rapidly as more and more developers continue to improve it. The HelenOS of yesterday is not the HelenOS of today and certainly not of tomorrow. Do not shoot yourself in the foot by dismissing your or someone else's contribution to it, or, even the project as a whole.
  • As a base operating system, HelenOS has the inherent problem with the visibility of its qualities. If a system, no matter how sophisticated and complex it may internally be, does not produce lots of noise and visual effects, it may be difficult to explain to someone that there is something interesting going on under the hood. This effect is reinforced by the fact that we are not composing the system from existing parts, but basically redesigning it from scratch. Be prepared to present your contribution to HelenOS and HelenOS itself in an appropriate way.
  • When your interviewer appears to be a dismissive one, try to use some of the argumentation from above. Also remember that interviewers are mere human beings with limited professional experience. It may happen that your interviewer will be Linus Torvalds or Jeff Bonwick himself, but chances for this are rather low. Even if, despite the odds, that turns out to be the case, you should still boldly go on with your HelenOS story. Which is more probable, your interviewer will have some experience with a couple of software components of some greater whole and thus will have a comparable set of experience as you (depending on the number of HelenOS projects standing behind you). There will also be cases when your interviewer will not be a match  for you and your HelenOS experience. The bottom line of this is not to let the interviewer dismiss your experience only because his company makes or contributes to some high profile software project, has some customers and earns lot of money.
  • Have you made your HelenOS contributions when you were a student? So what, this fact should not disqualify neither you nor your experience. To the contrary, participation in the development of HelenOS should have taught you to work in a loosely knit team of other developers and obey certain rules. Let's make this an advantage.
In closing, let me say that standing firmly behind your HelenOS achievements during an interview can have a positive effect on your employment and also on the general awareness about HelenOS. Why not use the leverage as efficiently as possible when it is available?

Tuesday, June 26, 2012

Who's fault is this?

Today I ran into an interesting phenomenon when writing a test application in Visual C++. The phenomenon appears to be a bug in Visual Studio 2008 optimizer, which seems to lose an update of a variable. Of course, it is equally possible that the bug is in my code, which does some really ugly stuff.

Consider the following function:

static Status test(int (__cdecl *f)(void *), void *arg,
    Status errstatus)
    Status stat = errstatus;

    __try {
        (void) f(arg);
    } __except((GetExceptionCode() == EXCEPTION_GUARD_PAGE) ?
        ((stat = SUCCESS), -1) : 1) {
        return (Status) -errstatus;

    return stat;
The idea of function test() is to return SUCCESS if and only if function f() triggers a guard page exception (and nothing else). The problem is that it does not work when using more aggressive optimizations.

Let's have a look at this on the assembly language level. Here are the relevant parts of function test(). At its beginning, we see register ESI loaded with the errstatus argument:

011C12C3  mov         esi,dword ptr [ebp+10h]
    Status stat = errstatus;
011C12C6  mov         dword ptr [ebp-1Ch],esi
The compiler moves the value to the stat variable, which lives at address EBP - 0x1c on the stack. The next interesting place to look at is the piece of code which updates stat in the case of the guard page exception:
011C12FF  cmp         eax,80000001h
011C1304  jne         11C1311h
011C1306  mov         dword ptr [ebp-1Ch],7D00h
011C130D  or          eax,0FFFFFFFFh
011C1310  ret             
011C1311  mov         eax,1
011C1316  ret
When the exception code, now stored in EAX is EXCEPTION_GUARD_PAGE, the stat variable on the stack is updated to 0x7D00 (SUCCESS). Because the __except expression evaluates to -1, execution of f() is resumed where it was previously interrupted. We'd expect f() to return to test() and test() to return the value now stored in stat. But looking at the end of function test() in the assembly, this is not how the compiler sees it:
    return stat;
011C12E1  mov         eax,esi
Instead of reading the stat variable from the stack, where the correct value is, the compiler returns the old value it previously loaded to ESI.

My impression is that either I am unintentionally doing something illegal, such as updating a local variable from the filter expression, or the compiler contains a bug which does not take the exception handling code into account when performing optimizations.