Lessons Learned From Migrating Old JS Codebase To React + Typescript Part 1

Part 1

For the last ~2 years one of the big part of my job was to migrate jQuery Frankenstein’s monster to something that can be actually maintaned. I can give some advice about how such a transition should be made if you’re facing similar situation in your project.

Working with legacy code can (and probably will) be painful. In my case it was more than 10k lines of code in unstructured files loaded without any bundler, no single source of truth, operating directly on DOM, no types, no test suite and no longer supported libraries loaded as script tag downloaded directly from author’s homepage. Sounds like a nightmare no matter if you’re junior, senior, backend or mobile developer.

I worked mainly on such codebase for the last ~2 years and one of the big part of this job was to migrate this Frankenstein’s monster to something that can be actually maintaned.

Unfortunately one of the most crucial project decisions was made before I started the job, but neverthless I can give some advice about how such a transition should and shouldn’t be made if you are facing similar situation in your project.

To be 100% clear - I was not tech lead or manager of some sorts, I was just developer with the most experience in JavaScript in the team, so I had a approximate vision of how our project’s frontend should look like.

Context

The application that I’ve worked on was a medium-sized ecommerce site with backend written on Symfony. The buisness purpose of the company that owned the site was to allow customers to use special offers, coupons or discounts on various affiliate shops and platforms such as Amazon or Otto. Each click or impression gave a fixed amount of money or percentage of client’s sum to pay.

Visitors saw a rather small piece of the entire codebase since client-side frontend was a small bit of the admin-side management features. 80% of interactive and most important features were not visible to the user. That doesn’t mean client-side frontend performance, scalability and maintainability was not crucial business requirement. Client-side frontend migration was also a challenge.

Focal point of the site was a complicated system of cross-validated list of forms with many edge cases and integrations with internal or external services providing scraped or manually entered discount codes. In addition to that were ordering functionalities and pasting from dynamically imported forms that aimed to make the entire experience as friendly and as possible.

I’ve mentioned cross validation - one input could be validated against - self - neighboring input - input of the form at the end of the list based on state of the other inputs in that form

And this is only the client side input validation. There was also submit-level validation that happened before submit, which had to bypass some of the fields in some cases and the backend that ran validation of the provided data against the data pulled from repository. So, there were 3 levels of validation but there was a fourth level - an edge cases where another administrator was working on the same set of data. Every message should’ve been clear as possible and indicating where is the error and what type of error it is.

This is a simplified diagram of data flow in the application. The most JS was in the second part, in so-called merchant forms

project-diagram

This is only a one part of many of the processes we were trying to implement in the project. Later in the article I’ll try to provide more examples of this.

Tech stack before

Before the idea of migration came up project frontend state was, to say at least, hard to grasp and maintain. For example, in one of the pages there was a list of ~20 files loaded just by including <script> in the twig template, each of them with number in filename indicating the desired order in which they should be loaded. Since there was no module system, those files were just giant IIFEs acting as a namespace.

Unit or functional testing was non existent, at least in the frontend domain. TypeScript was not used, same for linters and code style.

To be clear - for couple of years frontend wasn’t a focus of the project as much of the work was done purely in the backend. Also, there was no dedicated frontend/javascript developer to keep track of what’s going on.

Furthermore, there was no state management. Everything was done by direct DOM manipulation. I don’t have to say explicitly what it means. If you didn’t work with Javascript frameworks, it’s like having a clear separation of concerns in MVC pattern in framework like Symfony or .NET (language is not important here) vs. rendering everything inline with <?php echo ''. Debugging is a one nigthmare, but dealing with runtime errors in the browser is the other part of the story.

There was error monitoring service called Sentry https://sentry.io/welcome/ which, in theory should report any JS error in the client’s browser and send a big amount of data that could be helpful in debugging, but it was not properly configured.

As I said before there was no bundler (and module system) and some of script files were not available on Github/Npm and it was impossible to track dependencies in other way than CTRL+SHIFT+F *.js

CSS stuff was written in Less and was in good shape, since designer was working directly with stylesheets. There was some Gulp tasks running, but there wasn’t much to improve in that part of the project apart from integrating these Gulp tasks into Webpack build pipeline.

I’m not going to write about this whole migration process, so I’ll cut to the chase here.

Tech stack after migration

Most of the decisions about the process, technology, libraries and tools were made by me (and one other teammate who also had experience in JavaScript), based on my experience, research and analysis

Typescript +React with Redux in the dynamic part of administration panel. Redux-form was used to manage state of the form list, which eventually become the biggest mistake in the project.

Code structure was much clearer now with separation of admin and client bundles, with explicitly named directories. TypeScript module aliases were created to have shorter and more meaningful import statements

Webpack was wrapped with @symfony/encore which lead to the build-breaking errors few times. They were mostly related to differences in versions of the Encore, Webpack and TypeScript which required us to downgrade some of the dependencies.

Yarn was used with custom scripts with package.json to track all of the dependencies and their versions. Seems like a obvious thing to do, but not quite obvious few months before :)

TypeScript on the client frontend. We didn’t use React because that part of the code wasn’t dynamic and didn’t require complex rendering and/or state management. There was also problem with bundle size if we decided to use React.

Jest was used to unit test TS files and Enzyme to test React components.

Cypress was used for integration tests in most of the cases where admin interaction lead to the significant and important changes to the client side. Eventually we had ~300 test cases that took ~20 minutes to run on CI server.

I could write a book about problems we had with this test framework, but I’ll stick to the blog post for now, which I should release soon. I bet I forgot about something, but I guess this introduction was detailed enough to have a good impression what was going on in the project.

Lessons learned

Now, let’s talk about my advices about making such big changes in the project and what I learned this process.

Make sure decisions about frameworks, libraries and tools are not made by one person

In project I’m describing here, most of the crucial decisions regarding architecture, libraries and first steps of implementation was made by one contracted developer who left the company shortly after drafting some pull requests. I don’t have to tell you what are the implications of this kind of situation. As far as I know, he didn’t consult much with the bosses, so he had a free hand on this. When we took over, it was far from ready for production.

As a developer, I think that very decision that can impact project in some way should be brainstormed in one big meeting or few shorter sessions. Leaving big changes to one person can lead to issues in the future: too much opinionated take on code or relying on one person’s personal preferences. I’m not even writing about situation where someone just decides to use framework or library that no one in the team had heard of before.

I guess having a dedicated tech lead that chooses the technology solves the described problem but I still think that even senior or expert should make decisions somewhat based on the rest of the team skillset and preferences to benefit project in the long run.

Write and run tests before changing anything in the code

If you want to rebuild some modules of the project or (like in this case) almost entire frontend architecture, you have to be sure that everything works as expected. And you cannot base this on a assumptions (which in most cases are false), or words of someone in your team. Tests are not covering 100% of cases, but its a must-have in such a big undertaking.

We didn’t have integration tests from the beginning and it was a huuuuuuge mistake. There was just too many hidden interactions that were not tested. Having only unit tests in that case wouldn't be sufficient as there were too many interacting parts of code. Writing a good test suite covering a big portion of it would at least indicate problems straight away.

If you have dedicated testing team it’ll probably not be the problem. In my case, implementing tests was something that everyone knew should be done, but there were always excuses to not do it right know. Maybe in the future.

And when the future came, it was clear that we could've saved a hundreds of workhours if we implemented integration tests few months before.

Document everything possible

I know that nobody wants to do it, nobody likes to do it and <insert aint nobody got time for that meme here>

But documentation does not mean that you have to write 100 pages of .docx or Markdown.

Use JSDoc, PHPDoc or *Doc. Describe what it does in plain simple English.For some of you this is obvious, but believe me, not for everyone.

Documentation is also the commit messages. Use that commit description and describe the change if it’s not something obvious. This is gonna be lifesaver 2 months later when you try to decipher what and why this piece of code does that weird thing.

Of course in ideal programming world that won’t happen because every class, method or component is perfectly understandable at the first glance. But the real world is not some kind of tech talk or presentation at your favourite programming conference. You have to realize that people do make mistakes, have their own shorthands, and everyone writes code in a different way even if guided 100% by documentation and best practices. Writing that one sentence in commit message will save you alot of trouble. Believe me.

This can be a bit tricky if your project has a guidelines for git workflow such as number of commits per single pull requests. Also, not everyone likes atomic commits and long descriptions. If your project is close to deadline there are probably better things to do than writing detailed commit messages. But if that's not the case, I highly recommend trying approach described above.

Use your GitHub/GitLab/Bitbucket to create a wiki with FAQ and setup processes. This way you save a fellow teammate from being frequently annoyed by questions asked hundreds of times before. This is extremely helpful especially when working with full remote distributed team for obvious reasons.

All of the above are just good development practices no matter if you are migrating some huge project or just working on some simple feature. But in my case lot of this was either overlooked or nonexistent in our project.

Enhance progressively

By that I mean that you can’t jump straight into IDE and just rm -rf src/* and start rebuilding the whole thing.

Break one application view into small pieces. Then, break those pieces into even smaller pieces, that can be replaced without too much changes in connected files, classes, modules.

Start with small changes and enhancement. Extract this piece of code to separate component. Write test for that component. And when you are sure what it does, and what it should do - replace it with desired code. Don’t just yarn add angular for that one search input.

End of part 1

I hope this article is relevant to your work as a developer, will help you plan the work in advance and give you a chance to predict similar issues before they happen in the field.

In the next part:

  • Make the transition smooth
  • Consult with your team often
  • Be sure to have good arguments
  • Estimate wisely and carefully