Read here: http://johntate.org/node/231
My Genius wrote:So, for ages Linux has lagged behind Mac OS X and Windows. This used to be not so much in library compatibility, I can remember at one point using Caldera's Tarantella which came as a binary in 2001 (and was a remote admin type thing) and it ran on a the distros of the time. Back then, I was using Red-Hat 7.x. Linux matured on the desktop with Red-Hat 8.0 which was also when apt-get for RPM came around in those days before Red-Hat's own Yum. This is when Linux actually started heading down what some, even myself, have considered a road of no-return for compatibility.
Windows 95 and Windows 98 had a problem called DLL hell which was fixed with Windows 2000/XP. Windows NT avoided the problem by being out of reach from children and home users - and by being about as friendly as a street brawl. Though, I am sure with NT the problem was there - it isn't really important because I just need the analogy. Third-party applications would bring their own versions of DLLs which would replace Microsoft ones to add features. This became quite a problem when one third-party replacement replaced another and suddenly the earlier program broke.
Linux has a very different but similar problem, only it handles libraries as amicably as any modern operating system. Since the development of tools like apt-get and there being more packages than ever we have a well known problem of the downstream patches added to packages. This doesn't cause problems with one or more applications very often - though occasionally I've built things from source which don't expect certain downstream changes. The biggest bugs come from third-party developers that write to packaged library and these typically handle version changes well - the upstream developers handle this quite well - downstream patches added by the downstream can break third-party and proprietary applications.
Unlike DLL hell some of these bugs can get incredibly strange. Recently I've installed updates on my debian laptop for example that left it for a few weeks with a binary volume control. That is, either it is on or off. I don't know enough about the sound system but someone went and fixed that. Still, it isn't all a loss and getting rid of the downstream would reduce progress more than advance it. Linux has come a long a way!
Still, upstream and downstream problems will persist and this is mostly because of the law of unintended consequences. With large systems it becomes difficult to understand every component. This will happen on political systems, where politicans face unexpected disasters as a result of policy. This will happen in computer systems as well, where new downstream patches or major upstream changes will break things. There isn't much else to combat this, even my beloved FreeBSD has this problem with its ports occasionally (the difference being you can usually spot them while things are built and before installation).
Linux has quite a problem. What is the solution?
Downstream patches are great - but why just patches? There is room for something else which has advantages: shared objects that extend existing shared objects and applications exactly like plugins. An API and ABI for major components that are patched that can talk to new shared objects permits greater flexibility than patches for administrators needing to solve problems. To disable a downstream patch, you are going to need to apt-get the source and disable the patch and then rebuild. This works, but efficient it isn't.
With shared object plugins that are provided with an API by the upstream it allows a new host of possibilities. Firstly, downstream bugs provided by them can be disabled by unloading them. Secondly, third-party software has the stable set of core libraries that come with the system. Thirdly, there can suddenly be something new entirely: third party extensions to Linux. These can be in the form of usability extensions, things specific from hardware developers, and other things. The great thing about C and C++ is being able to use shared objects like this but really that has not matured.
To do it what is needed is a standard way for all kinds of applications to support plugins in this format. This needs to be implemented into the upstream which is quite a task of both engineering and software politics. Rome was not built in a day, and so this doesn't need to occur with every application at once. Slowly though it would be possible to provide an interface to APIs and ABIs which provides the extension of them. A good start would be in any software package where downstream patches become immense in volume and lead to bugs, where patches can be replaced by these shared object extensions/plugins.
I'll have to start hunting for the perfect candidate. Producing a working example would be the best start, especially at getting a picture of how it can be made universal in structure. For many applications, a C++ class could be linked in as a static object that provides classes for these extensions to use. A universal approach would be the long-term goal which can be copied and implemented into a great deal of software.
This does not just solve a repeating problem of downstream hell, but it adds an innovation which so far no other operating system has: openness on a binary and source level. Third-party changes like this on Windows in its entirety (and not just a few things) might never happen. On Linux, it is a good area to experiment. Also, the interfaces to do it can have their own API, which can allow for extensions to be built without the headers code for anything more than the extension API. Because of the ABI provided in the binaries I've envisioned this can work.