Thoughts on rebuilding the Umbraco CMS back office
I've had this post brewing for a long time and tried several versions of it. If your reading this then hurray this one must have been "good enough". I hope the following ramblings make some sense. Be warned, this is a long one but it needs to be.
Umbraco CMS currently has never been stronger. Version 7 is absolutely amazing and rock solid. These are golden days that we live in as an Umbraco developer, agency and for our editors. Bravo. Its a testament of just how far it has come on that I raise the thorny issue of rebuilding a large chunk of it, are we getting to the point where we need to think about rebuilding the back office? I know right, what am I thinking? That makes no sense, I just said it was rock solid so why rebuild it? Hear me out though.
This is a bit of a controversial topic as it means change and as humans we don't cope with change all that well. So let me quickly cross off some of the obvious queries and knee jerk reactions you might have before we get into the meat of this thought experiment to try to calm some nerves and sent the scene:
Its a task for HQ but as a community we can start and shape the discussion, after all its us that will be using it.
We don't have to rebuild the back office at all if we don't want to!
Its a really big job and I don't expect anything to start on it for 18 months at least
Due to above lead time I'm not suggesting any tech choice at all at this time, just ideas. This post is framework agnostic on purpose.
I know you have probably invested heavily in learning Angular and all that jazz and don't want to lose that knowledge or have the heart to retool, I get that.
The current back office has been amazing and will continue to be for some years yet
Right, that is the obvious stuff covered. You all good? Great, lets do some more digging and see why I and others are starting to murmur about a rebuild, whats wrong with the current one and what benefits we might get from it.
Very short history lesson
Honestly this is the short version but a little history is needed so we know how we got here and where we might go. The current back office is in its 5th year now, I remember watching Per Plough talking in a key note about Belle (the code name for the new look back office) at Las Vegas at the first uWestFest. In that talk he explained the choice for using Angular, the departure from .net for making custom backend widgets and property editors (woohoo! good bye user controls).
Umbraco as it is is stuck on Angular version 1.1.5 (or there about) with rumours of it even having some customisation in there too...but of course it has. Angular moved on of course but for some reason the Back office didn't (I suspect they changed the API at some point which broke some stuff before a big release). Then something odd happened within the Angular development team.
Other frameworks started coming out, some really big apps were made with Angular, new ideas got added to the mixing pot, lessons were learnt. To fix these issues and make changes they decided to completely rewrite it...over several years...with no backwards compatibility. It didn't go down well from those that had invested in the framework, everyone was now in effect in a technology dead end, including of course Umbraco. Angular v2 (actually v5 I believe...no not THAT v5) is out and the version Umbraco is on is now referred to as AngularJS. Confused? You should be, its a bit of a mess. There are ways to allow for upgrading to the latest Angular framework but its nothing really simple so although this might sound like an option it really just muddies the waters even more in my opinion.
What is wrong with mature tech?
So we have a framework that has done us proud for 5 years, what is wrong with that? Why would we want to change it when it sounds like its working?
This means that we "mostly" have a separated API back office from the front-end back office. As a result this means that the investment in development effort to switch from Umbraco V6 to V7 would not need to be repeated should we chose to switch out the front-end of the back office in the future. Rather than a complete rewrite we only need change the front-end layer and continue to use the existing API's if that makes sense (I am of course aware its a huge task and that sentence makes it sound like its easy).
We could just keep using it as it is. It works right? Its certainly an option and to be clear we will have to until a future replacement is ready anyway assuming we rebuild at all. But there is a danger that its not just the tech that becomes legacy but the knowledge of how to do it too. Currently getting docs on AngularJS v1.1 is not easy or obvious, there are not many Stackover flow questions flying around for the version we are on, the knowledge is hard to find/attain and is not all that transferable. Umbraco training is the only place you can get a few hours of hands on time in AngularJS. New front-end developers coming to Umbraco are going to have a hard time of it and also be amazed it doesn't do some of the cool stuff they just take for granted these day. Dare I say that it could be that AngularJS is becoming the the XSLT of previous versions? It did the job but folks found it old, quirky and odd (unlike Razor which is just quirky and odd).
Looking under the hood
I've recently been doing a lot of digging into the back office as part of a optimisation project for a client and if honest I had an itch to scratch. Its the sort of digging most won't do as really we only normally have to build property editors and dashboard but when you start digging it sort of becomes clear that a lot of the core of the back office, the really deep stuff has the feel that it is still pretty much like it was 5 years ago when the first prototype was built. Its got that "all best intentions are to come back to sort this out once we've got this prototype up and running" feel about it, you know, the ones you never actually get the chance to come back to.
This is not a direct criticism (we all have code like that), what is there works really well and enabled all this in the first place, but imagine if we could re-visit it and get it bang on rather than "it works". Then rather than the black magic feel that it has at the minute others would be able to dig into it more, pull requests would be easier, code layout would be better, services would be obvious and standardised. There is a benefit right there but you'd need a reason to go in there and do that, a rebuild (or retool) would be the perfect time. As is I suspect few dare go too deep (even in HQ) and poke it from a distance with sticks. This is of course just my opinion/judgement from looking around it, I could be wrong.
Some of you I know use Angular on the front end of you websites and the fact you can use it both in the back office of Umbraco and on your client sites is fantastic. I'd get why you wouldn't want to change the back office tech. But again time doesn't stand still and finding staff who can pick up a site or a widget or a package is starting to become a problem, because you won't relearn a tech they have to learn an old one that is in a dead end.
By updating the tech we get to tap into the existing knowledge of new hires rather than have to train them up. Again we've had 5 years out of AngularJS in Umbraco, that is not a bad run now is it?
What about all those packages we've written, if you rebuild won't we lose those? Well yes, yes you will. But you'll have plenty of heads up that its coming and in the meantime we will have had V8 and maybe even V9 by then which will have had their own share of breaking changes. If the packages are good enough then there is time to convert them over. Luckily some of the biggest packages are already in Core (hello Nested Content) so would get converted anyway, "most" others don't go into crazy depths with Angular or if they do they tend to use stuff in the Core that can change between versions and break anyway (Angular "tweaks" are never considered a breaking change by HQ it seems).
Wait, I mentioned an iframe escape hatch for old legacy code from v6 a while back, could we not do something like that for the older AngularJS stuff? Hell no! How long do you think your code should stick around for? V8 is the "clean up" version were we get to dump a lot of really old legacy code, it would be a damn shame to then straight away start a fork of the back office and try to run two versions of it at once. This is meant to be a clean up and a rebuild not trying to run multiple out dated frameworks in one. The biggest code to be changed will be the Core code that runs the back office, that to me is where the biggest gains can be had for performance, maintainability, consistency and extensibility. Older packages will simply have to be updated or retired. If the are that good then others will upgrade them and bring them into the new style if you don't have the heart or time so fear not.
I'd draw a line under old AngularJS code if it was me but then its not me in charge is it? So yes you "could" run your property editor code in whatever new framework we use, I'm sure we could add a bunch of shims and proxies to make it work one way or another its just not pretty and I don't think its needed.
What do we actually "develop" in the back office?
When there is talk of a rebuild or "we should use framework X in the back office" discussions there are some that think that this would cause all their skills to be scrapped and a massive effort to relearn new technology (or yet another framework). Thing is, what do we actually "do" in the back office as front-end developers? With a few notable exceptions most of us build a fancy property type, which might need to make an API call to either Umbraco or a 3rd party end point, maybe a dash board, create a new section, maybe add a menu option. That is it really, that is 95% of what we might want to do.
How do we develop for the back office?
The core of the back office on the other hand and if you need to "go deeper" should use tech like WebPack as the gains would benefit everyone. If you want to work on core we'd have a guide on how to get setup with WebPack et al and a config file already written for you so it all just "works" without you having to do much other than have Node installed and follow a few one time command line instructions (or like we do just run a .bat file). This is about getting gains for everyone not just JS whizz kids who like playing with fancy stuff, that won't help any of us now will it?
How might we do it?
HQ would have to do this one, this is not one the community could do. But as a community we can help shape what we would want it to go like and let HQ know that its something we want doing and focusing on (assuming it is).
I really like the current look and feel, I don't think there is any reason to depart from it currently and it would only add more work to what would already be a bit job. I'd suggest that on the whole we leave it as is and just rebuild the tech under it.
I'd suggest the ideas of what it should be like are fleshed out between now and then and that the intentions to rebuild are announced with a rough version number to target for. Right now we could do a bit of optimisation work in the existing back office code to tidy it up a bit and get it fit for the next 2 years. Then it can hold the fort while the new rebuild beings and is completed along side it. This is very similar to how we would rebuild a large website, shore up the existing site to buy us time while we work on the new one along side it, once ready swap it out. Again I think starting this is about 18 months off given HQ's apparent current work load and aims but seeing the new road map at Code Garden might give us a better indication of time. If we shout about it enough they might pull it forward but I think starting it in 18 months or so is about right for a target.
Then ideally the new back office gets released as V9 or possible V10 (which I suspect won't be called that and will be Version X or something, that seems to be the in thing to do these days) and there will be much rejoicing.
There is much more to discus on this so I've created an Umbraco issue (link is at the bottom so you don't get distracted) so our discussions don't get lost on twitter. I'm also keep to chat this through with anyone at Code Garden in an Open Space or at the bar :)
We should use [insert framework here]!
No we shouldn't. Right now is not the time to pick a framework. Right now we should be discussing what we want the back office to do for us in 2 years time and then see what fits. Anything could happen in that time frame. Even me mentioning WebPack above was a bit naughty, it makes it sound like its a given but it was only for example. Just because its hot now doesn't mean it will still be at the time when we break ground on a rebuild in 18-24 months time.
The danger of picking or discussing a framework now is it can cause Confirmation Bias in others or even the Halo Effect which can cause us to miss out on a true best fit for all. By naming tech now we are in danger of framing the discussion about the tech rather than what it is we want the back office to do for us regardless of tech. Let's instead talk about the changes we would like to see, the gains we would want to have, the friction we would like to ease and the legitimate worries that we would have to tackle to ensure current users can adopt any future change, new users can pick it up quickly and give it a strong foundation to take Umbraco forward for another 5 years.
Its joked that frame works pop up all too often but in reality they are quite rare beast. Yes you get some pet projects or side projects cropping up but there are a lot less than you might think that take hold and stick around. AngularJS has done the Umbraco project amazingly well and will continue to do so for another 18-24 months with ease while hopefully we work towards a more up to date solution however now is the time to be thinking about succession. Now is the time we can talk about as a community how we might want that new front-end setup to do, look and feel like, effectively so we have a job description before we start interviewing candidates for the roll.
I'm very excited that this is a discussion we are having now, it once again shows just how far Umbraco has matured and how rock solid it is that we can plan for features 2 years down the road. Bring it on.
To comment on this post and share your ideas please head over to the Umbraco Issue Tracker, see you there.