Changing our application development process

[Versión en castellano]This link opens in a popup window

Backgrounds

Not long ago, our development process was an antThis link opens in a popup window based one, managing dependencies in a not very good way and using a series of custom scripts to prepare and deliver product and project bundles. Merging was really a nightmare, and we lost a lot of time making manual tasks and training new colleagues in our way of doing things. As time passes, I wonder over and over how we could work that way.

vuelta-al-bloque-cubo-madera-icono-bombilla-simbolo-humano-cabeza-no-tiene-ideavuelta-al-bloque-cubo-madera-icono-bombilla-simbolo-humano-cabeza-no-tiene-idea

I any case, we decided to move forward, changing everything upside down. This involved changing our versioning system and mainly our building and development process. We changed a lot of things, and it was not easy, indeed not easy at all. We needed to think through the important and no so important matters, and how we could improve new colleagues onboarding.

What am I going to talk about?

In the next points I’m going to focus on the different issues we had to address

  • Changing the versioning system, from Subversion to Git
  • Modifying our CI and CD processes, improving our dependency management solution.
  • Modifying development process, hiding complexities.

Changing the versioning system

As a matter of fact, changing the versioning system isn’t a task at all. It’s just a change of tool.

11668623_2094522711668623_20945227

Even though that’s right, I would like to highlight some points, just if you are in the same situation,

  • We opted in to use Git, not for it being the best tool in the market or for doing everything perfectly, but for being the de facto market standard. There are somethings that I like and other that I detest although I’ve learnt to live with them.
  • Main problem when changing tool is training, training your people in using a new tool, and forgetting about old manners.
  • I realize that I had some big expectations about everybody using Git. Although a lot of people know about it, a lot of companies are still working with Subversion, and a lot of youngsters don’t know what versioning is, and what it intends to solve.

In any case I don’t regret moving on to Git, it was a step, a necessary one, in the change we were trying to address, improving our building process,

Modifying our CI and CD process

The best thing that I can say about how we were used to working is that it was a home-based solution, point. Dependency management was so bad done, that I don’t know if I dare to say that we had it at all. Bundling our projects was based on shell scripts, variable substitutions and all those things. It worked, but working is not enough, it must be maintainable, it must be repeatable and as new people came to work with us, replacing old ways was a necessity.

21683340_Teamwork of tiny people with gears and cogwheels21683340_Teamwork of tiny people with gears and cogwheels

If we talk about tools, the change seems not to be so complex, we just began by using Nexus Repository, Gitlab CI process, integrating SonarQube and delivering the bundles through Docker. Quite simple and standard, isn’t it?

Actually, it’s simple at all since, under the hood, there are many processes involved, changes that must be done, and issues that have to be solved in some way, to be redone in a different one in a few weeks or months, after the process is consolidated. So, our change was:

  • Planned, we had a focus, and we knew our target although the process was not so straightforward as we wanted it to be.
  • Iterative, integrating subtle and small changes in how we worked. At the end, we were able to be where we want to be, give or take, from the very beginning.
  • Global since changes affect everything. Do you think that we could talk about the CI and CD without talking about changing the building/development process?

The change in our development process

Without any doubt, of all the tasks we performed this was the toughest one. We didn’t want to follow a cosmetic approach since what we were actually trying to solve was not only improving the CI process but ensuring a productivity increase for both our developers and operation teams.

developmentdevelopment

We changed the tool, forgetting about ant and moving on to GradleThis link opens in a popup window. But, believe me, the problem was not changing the tool, but the things we did with it. Just to highlight some of them:

  • Avoiding manual deploying on a local application server. Teaching every new person, even Frontend guys, how a Java Application Sever works is time-demanding and frustrating, both for the trainer and the trainees.
  • Simplifying merge process. Isolating properly between global libraries, global messages, global pages and scripts, and project ones.
  • Restructuring our build scripts, bundling common tasks in a local Gradle distribution. Improving the whole-project compiling packaging, which includes server-side Java Code, REACT TypeScript frontend based one.
  • Improving reusability, the same bundling process is used both by our developers and by our CI process.

Some things we learnt

Changing is not easy, but that doesn’t mean that it’s not necessary. Doing always the same, only drives to stagnation, and stagnation only drives to disappearance.

Just to give you some figures, changing everything took us about 1 work year, about one person and a half, we learnt a lot, but mainly we improved a lot, and we now that we have still a lot of work to do.

Embrace change, don’t hesitate to do the things that must be done.