Monday, November 23, 2009

Two busy weeks - back from Milan - Number portability - the Belgium case

Last Monday, I did a one day trip to Milan to give a class about our middleware for number portability. A big piece of Java work that would have very much benefited of Spring.

In fact I solved the beans configuration exactly the reversed way Spring does. Basically I stored all the configuration stuff in a single object for each sub process and every component take what's relevant for its job.

I know this make all this code dependent on this big fat object. I think some people call that a white board design. I agree this lack elegance and hinder re-usability. On the other side it simplified very much configuration and because objects share the config info it is naturally consistent. The same object also implemented a factory pattern, creating and initializing most of the objects (a better part of my design).

Most of us keep looking forward, but I think checking what happened backwards brings also some values.


There is a lot to say on this project not only on the Java side but also project wise.

Number portability (moving from one operator to another but keeping your number) has been implemented in most European countries (because of European market regulation). Yet each country had the freedom to implement it its own way. This has been for me an opportunity to work many times ( for Belgium, Luxembourg, Spain, Portugal, Italy, Germany, Hungary Finland, Latvia, Nederlands, Lithuania, Estonia, Croatia, Hungary ) on similar but different projects.

Typically, number portability is solved by a central database that lists those numbers that have been ported to another operator. An administrative process - kind of work flow -  has to be implemented, usually with time constraints. Once porting is agreed between the donor and the receiving operators, a technical phase propagates the porting information to all operators. The net result being that all operators own a copy of the central database that feeds the operators switches (usually trough what is called an Intelligent Network platform).

Belgium was the first process I developed. The central database project was outsourced to a contractor -  a very  big, well known international software house.

The entire process was driven by a committee piloting the contractor. That generated what committees like to get: papers (probably also a lot of meetings and a high volume of coffee breaks).

The full documentation filled an entire CD (for comparison I received the Spain technical documentation - a single file format description - on one page).

Technically this was my first exposure to Web services (more or less 6-7 years ago). That part was great. However the web services was described as an XML schema - no WSDL. So I had to use an XML parser,code and decode the messages.

The rest of the project was 'nuts' and was a school case on what not to do.

The committee-contractor tandem generated a lengthy and mostly absurd acceptance process, so we had to simulate business transactions of all kinds in an absurd repetitive fashion. Testing and preparing acceptance reports took finally 3-4 times more resources than developing the Web services.

More funny was the total absence of service oriented attitude of the contractor. The idea of giving an easy test platform, a reference implementation, or any basic tools (e.g: raw messages) for developers was out of their mind. Just for one reason. Not only they got the contract for the central database, they also expected to take the business of all the connecting operators. So the motivation to help anybody was naturally extremely low.


They also provided a web based application. They used Java and the most modern computers stuff at that time - multi-processors Sun servers, clusters, Oracle, Nokia Checkpoint appliances, load-balancers etc This was very professional. Unfortunately the application quickly turned out to be a pig. Users started to complain from availability, performance , error messages, failling transactions etc.

In my opinion, this was due to the inexperience of the programming staff and finally a very poor design.

A constant problem of big consulting companies is that they send their senior people to win a deal and send junior staff to implement it. As the amount of problems became important, their support staff could not follow anymore. They entered into crisis management by the book - thus delivering an help-desk procedure that requested a complete network description just for asking a password... Of course, this exasperated their users more than it solved problems.

I never attended these comittee meetings. The specs were done when  I started the project for my customer.Why have they decided to build their on messaging infrastructure on top of relational database using Soap. I don't know. However I saw later similar designs and it seems they copied a system developed in the UK.


In a search to create additional work, they decided that a VPN was not enough. SSL certificates were used between all parties. Good. Obviously they created their own certificate request process with conventions about what to put in each field of the certificate - this was described in a 20+ pages document (all other countries I saw after, simply ask to exchange public certificates between the operator and the central database - final point). In fact their policy and certificate distribution could have been meaningful if the transaction had to be secured end to end - operator to operator - which they did not.

In the line with their policy to create work, the certificates were issued for a single year, this created a permanent state of maintenance multiplied by two (test and production had to be renewed). The funniest thing was when I setup our messages reception point. The only way I could setup the server to get messages was turning off client certificate authentication! Half of their stuff did not work because they were not providing a complete certificate chain. When I discussed that with the contractor security 'expert', I could measure a long silence that finished by  'what's important is that it works now'. Indeed but  as far security is concerned...

A few years later, the outsourcing contract expired. The operators committee took the decision to swap the contractor and the entire application! So the new contractor re-implemented the entire system ! Their implementation had to be identical for the Soap transaction and they decided to keep  the SSL security system, unchanged...

No comments: