I recently needed to sync two databases with similar content but different structures. Here is how I solved it, and what I discovered along the way.

To illustrate I will use the objects below.

const sourceUser = {
    id: "94aef3ae-43d1-4847-b5c3-c2ea2d42c52a",
    email: "falk@liip.ch",
    rights: ["read", "write", "admin"],
    otherStuff: "whatever",
};

const targetUser = {
    id: "3",
    mail: "falk@liip.ch",
    rights: ["read", "admin", "write"],
};

Constraints

I had three constraints for my solution:

  • It should only use plain objects. No Sets, Maps or Classes. This keeps things simple if an API sits between.
  • The pattern should easily apply to any object once set up.
  • It should be built in TypeScript since the rest of the project uses TypeScript.

Let's get right into it.

Mapping the Structure

The first step was to map the sourceUser into the same structure as the targetUser.

const mapUser = ({ email, rights }) => ({ mail: email, rights });

const mappedSourceUser = mapUser(sourceUser);
// {
//    mail: "falk@liip.ch",
//    rights: ["read", "write", "admin"],
// };

I could also have built a configuration based solution like below.

mappings:
    users:
        email: mail
        rights: rights

But in practice, I found it is often easier to just use code. In most cases processing values (like toLowerCase) is needed at some point. At that point you will be better off with just code. The guiding principle here is plan for change.

Comparing Objects

Then I structurally compared the objects.

const hasChanges = !equal(mappedSourceUser, targetUser);

I chose structural comparison here, because it nicely fits the goal of easy application for any object.

Full Update on Change

Whenever a change was detected by the comparison, I updated the whole user in the target database.

I opted for full updates on purpose. While this is less efficient than partial updates, it also makes the system a whole lot simpler. The guiding principle here is KISS (Keep It Simple, Stupid).

Easy enough, right? Here come the problems.

Problems

The first problem is that targetUser contains an additional field id that mappedSourceUser does not have. This means a naive comparison would always find differences, even when the relevant data is identical. We only want to compare fields that are relevant.

The second problem involves the rights array. It is not sorted, but the order does not matter for our purposes. The arrays ["read", "write", "admin"] and ["read", "admin", "write"] should be considered equal.

Finally, there is the question of debugging. How would I troubleshoot this setup if a comparison always flags objects as different? I would need some way to see which fields differ and why.

Conclusion

So basically I need a comparison library that:

  • allows picking fields to compare
  • ignores order of certain arrays
  • logs readable diffs for debugging

Surely something like this exists, right?

Not to my knowledge. That is why I built sameish, a lightweight library that does exactly that. It works in both browser and server environments, with an optimized bundle size of 1.57 kB gzipped. The library is fully type-safe, including the picking of fields to compare.

You can try it out on the playground.