Simon Brown - architecture tag

QCon London 2008 - day 2

Fri, 14 Mar 2008 08:46:00 GMT

The first session I attended on day 2 was called "Architecting for Performance and Scalability", where representatives from Terracotta, (Oracle) Coherence, GigaSpaces, etc (and eBay) came together to talk about the different approaches to building scalable systems. It was surprisingly civilised and it was interesting to compare and contrast each vendor's approach to dealing with the scalability problem. Here are my takeaway points from this session :

On average, the audience thought that they could squeeze 2x performance out of their existing systems if given a couple of weeks to tune it. If you need any more, you need to architect/design with that in mind. In other words, you need to design in scalability from the start.
Ultimately, many of these decisions are economic in nature. It's okay for a business to demand ultra high scalability, but they will pay for it in terms of hard cash and time to market. It's all about trade-offs.
Continuous performance testing is an important thing to do if you're building systems that are performance critical. The implication of this, of course, is that you *have* something to test, which for me points to an iterative development approach and an executable reference architecture as early as possible.

The next session I attended was called "A Tale of Two Systems", which basically presented a picture of what happens when you do and don't design your software. Anybody experienced in software development won't have seen any surprises here, but it was nice to see the good and the bad contrasted in a very down-to-earth way. There was a definite agile spin of all of this; with talk of flat team structures and a distribution of the design responsibility throughout the team. In fact, Pete stated that "he'd never worked on a project that needed an architect". While these approaches work well for small and/or simple projects, I'm still of the opinion that *some* architecture needs to be performed up-front and that somebody needs to take ultimate responsibility.

Following the talk about software design was a talk about user interface design, entitled "User Interfaces: Meeting the challenge of simplicity". This session looked at the art of designing user interfaces so that they appear simple to the user, and that how making even the smallest of changes can have a huge impact. One of the most interesting parts of this session was that it almost completely paralleled the session that preceded it; in terms of talking about agile development, feedback, simplicity, you aren't going to need it (YAGNI), etc.

The final session I attended was Neil Gafter's look at the new features that are being considered for Java 7 and beyond. I've not been following this too closely and it was interesting to catch up with it all. One of the things that struck me most was that the Java platform JSR hasn't even been started yet and that Sun don't seem to have enough resources to do everything that they want to (apparently JavaFX is more important?). I was under the impression that major releases of the platform were going to be on an 18 month cycle, but clearly that's not going to happen. I also don't necessarily understand where/how the open source stuff fits into all of this. There are some nice smaller features being considered for Java 7 (multi exception catching, easy exception rethrowing, the ability to switch on Strings, etc) but part of me thinks that maybe the bigger language changes (e.g. closures) shouldn't be implemented. Perhaps it might be better to stop making big changes to the Java language and start putting more effort into something else (e.g. Scala, Groovy, etc).

All in all, another great day at the conference and some interesting discussion in the bar afterwards.

QCon London 2008 - day 1

Thu, 13 Mar 2008 09:32:00 GMT

Erich Gamma's keynote got the QCon London 2008 conference underway, where he talked about his experiences with the Eclipse project over the past seven years. There were a few interesting points during this session but I'm going to write about those separately. The thing I will say is that it's always good to hear the real-world stories about iterative development - in this case, the Eclipse team work on 6 week iterations and don't necessarily have a fully shippable product at the end of them. Erich wrapped up the keynote with a demo of the new Jazz platform, which pulls together all of the tools in your standard development suite into a fully integrated workflow. This looks like a rehash and enhancement of Rational's Unified Change Management (UCM) platform using Eclipse as the front-end. That might not be totally accurate (I think I saw a Subversion bridge in the slides), but you get the point. Certainly worth a look, although the "commercial open source" thing confused me.

As for the main sessions, I stuck to the banking track for the first day and that turned out to be a good decision because there were some really interesting presentations.

First up was John Davies who jumped in at the last minute because the speaker for that slot couldn't make it to the conference. Instead of a session about domain specific languages, John presented an overview of technology within the investment banking space. It was a really interesting talk and very nicely summarised many of the trends that we've seen over the past few years (e.g. compressed on the wire message formats rather than XML, etc). The key takeaway point for me was that you need to design for scalability. This is one of the reasons why I think it's important that software systems have an explicit and intentional architecture, with somebody taking responsibility for it.

Next up was Iain Mortimer who presented a session that was entitled "Keeping 99.95% uptime on 400+ key systems at Merrill". Although this was a relatively interesting session, I do think that the title was misleading. Instead of talking about how the 99.95% availability target was being satisfied, Iain talked about how Merrill monitored those systems, and particularly about the rules that they used to monitor those systems. Iain said that their availability requirements allow for 18 seconds of downtime per day, but didn't go into detail about any failover or recovery techniques that allowed them to meet that goal. So in that respect, the session was a little disappointing, although it was interesting to hear about the challenges of monitoring 400 systems across a global organisation in a consistent way. The key takeaway point from this session was that you need to design systems with monitoring in mind, which I completely agree with.

Third on the track was Bertrand Delsart from Sun, talking about the real-time Java specification. Although there were a couple of banking examples thrown in, this session was essentially a generic RTJS talk. The closest I've got to real-time Java is BEA's JRockit JVM with deterministic garbage collection but, as Betrand said, garbage collection is only one part of the story - Java apps also suffer jitter from the JIT compiler kicking in at unwanted times, etc. While this isn't something I'll probably try out myself (Sun's real-time Java VM only runs on Solaris 10 at the moment), what they've done is built a framework onto which you can build your applications where you decide which parts of it are regular Java, soft real-time or hard-real time. My understanding is that the hard real-time stuff is made possible by utilising the underlying OS real-time threading and some clever use of non-heap memory spaces, in addition to appropriately scheduling the garbage collector so that it doesn't interfere. Cool stuff and I think we'll be seeing this pop up in the banking industry soon.

Next was Betfair talking about their new Tradefair platform and some of the challenges that they need to overcome to provide a highly scalable, highly available trading platform. Again, there was some interesting discussion of the problems and high-level solutions, although many people (myself included) came out of the session not really understanding what they had done. They were very sketchy with the details and I'm left wondering why they couldn't have implemented their system using something like JavaSpaces (what they described sounded like a JavaSpace - put many things in and match them up). The thing I did like about the session was their openness in admitting that none of the solutions were ideal (all had trade-offs) so they had to pick the one that fitted their needs the most.

The last session of the day was a talk about LiquidityHub, a new trading platform that was built from scratch in just nine months. As a disclaimer, I work at Detica and have many friends that worked on the project. Having said that, this was one of the best sessions of the day. It presented an overview of the business problem, an overview of the chosen architecture and a look at how some of the technologies were used to build the platform. This project shows that it is possible to build a high volume, low latency platform with mainstream Java-based technologies. BEA's JRockit JVM was used to reduce the jitter of the Java runtime, making it possible to achieve a service level agreement stating that messages should pass through the platform in under 100ms. With its good coverage of everything from the business problem down to some of the implementation details, this was a great way to end the first day.

When will the offshore bubble burst?

Thu, 14 Feb 2008 11:08:00 GMT

Hearing management teams talk about offshore is becoming a more frequent occurrence these days and most large organisations that I've met in the past six months are starting to invest in offshoring if they've not done so already. That's not really surprising when you take a holistic view of the economics, but I'm convinced that the current offshore bubble will burst in the next 1-2 years, making way for a much more collaborative model. Let me explain.

Most organisations see offshoring as a way to reduce the costs associated with software development. The basic premise is that "software development is a commodity" and that "commodity development is low risk and can be easily offshored". Effectively, organisations are taking a "software factory" approach to software development where they throw specifications/contracts at offshore teams and expect working software in return. There are a couple of points that I want to pick up on here.

Software development is a commodity : I've heard this a lot recently, from organisations on the buy-side and the sell-side (e.g. consulting companies and system integrators), but I don't agree with it. Software development is still a complex discipline and you can see this by taking a look at the rate of technology change driven by people trying to make development easier. Just look at how many Java web application frameworks are available. Sure, there are parts of software that can be and have been commoditised, but there's still a lot of bespoke development that goes around it.
Offshore reduces costs : The most compelling reason for organisations looking to adopt offshore is that "the daily rates are much lower for offshore resources than onshore resources" and if you compare them in terms of cold hard cash, you can't argue with it. However, I don't think that organisations are thinking about the additional investments that they need to make in order to give the current offshore model a chance of success.

For example, it's fairly safe to say that most organisations work in a fairly ad hoc manner and need to put some investment into improving their processes to enable offshore development to work. This might include improving the way in which requirements are captured, prioritised and documented before even thinking about how best to structure the overall solution and define the work packages/specifications that will be sent offshore. Plus, you then have to factor in the additional cost of testing that will undoubtedly take place when the end-product has been developed and the potential latencies that will be introduced from teams being distributed across the globe. Sending work offshore introduces work onshore to manage it.

If you're reading between the lines, you might be thinking that I think offshoring is a bad idea, but that's not the case. The same problems apply regardless of where the "external" team is based because you've artificially introduced barriers that prevent the teams from successfully working together. Offshoring just makes it harder because of differences between time zones, cultures, etc. What you have here is a distributed waterfall model of software development.

Requirements : The buy-side organisation figures out what they want, scopes it and defines it.
Analysis : The buy-side organisation analyses the problem and starts designing a high-level solution. Anything from a small component to the full solution is documented in enough detail so that an external team can understand and deliver the solution. I've even heard of teams that take this process down to the pseudocode level!
Design : The offshore team take the analysis and start designing software.
Implementation : The offshore team implement and unit test their code.
Test : Depending on what they are developing (i.e. a component or a whole system), the offshore team performs some higher level testing before shipping the deliverables back to the buy-side for further system and integration testing. The usual quality assurance/bug fix cycles commence, with enhancements and change requests being sent offshore.

Our experience in software development has proved that the waterfall model doesn't work in many cases, for a number of reasons. These range from not delivering a fit for purpose solution because the requirements change once the design is "signed off" to actually failing to deliver at all. Plus, from an architecture perspective, handing-off your architecture to the development team also reduces your chance of a successful project. I've been called in to look at a lot of poorly performing, unscalable, etc solutions recently where the architecture hasn't been thought out well enough. The software factory approach to software development needs well defined signed-off specifications for it to work, but that's the one aspect of software development that we know just isn't possible (in the commercial world, anyway). If organisations can't get software development right using this process with an onshore team, how are they going to get that same process to work with an offshore team? Offshoring has the ability to reduce costs, but you're making some trade-offs.

No, the real problem here isn't offshoring, it's the engagement model that organisations are using. I think that, for offshoring to work, the engagement model needs to move away from the distributed waterfall and towards something that is more inclusive, collaborative and iterative. Agile methods bring everybody together and focus their efforts towards a common goal, which is the same thing that needs to happen with offshoring. Offshore teams need to be fully brought in to the overall software project that they are helping to build and the focus needs to switch from seeing them as a service provider to seeing them as a part of the team. More involvement and more collaboration is what I think will help make offshoring a successful approach. In fact, let's not even use the term "offshoring", let's just think about this as a distributed project team.

To wrap-up; I don't think that the current "software factory" approach being adopted by most organisations is healthy or sustainable, and organisations are starting to become dissatisfied with the results they are getting. Waterfall approaches have failed in the past, what's different this time?