In the beginning, there was theory

Architecture students at the ETH learn this so-called "coaster method" in real estate economics classes. Planning and building a house of any size is no easy task to begin with, and neither is understanding the financial aspect of it. To understand all of those calculations, some students created spreadsheets that do the calculations for them. This is prone to error. There are many questions that can be answered and many parameters that influence those answers. The ETH IÖ app was designed to teach students about the complex correlations of different factors that influence the decision. Furthermore, if building a house on a certain lot is financially feasible or not.

The product owner at ETH, a lecturer for real estate economics, took the tome to create such speadsheets, much like the students. These spreadsheets contained all calculations and formulas that were part of the course, as well as some sample calculations. After a thorough analysis of the spreadsheet, we came up with a total of about 60 standalone values that could be adjusted by the user, as well as about 45 subsequent formulas that used those values and other formulas to yield yet another value.

60 values and 45 subsequent formulas, all of them calculated on a coaster. Implementing this over several components would end up in a mess. We needed to abstract this away somehow.

Exploring the technologies

The framework we chose to build the frontend application with, was Vue. We used Vue to build a prototype already. so we figured we could reuse some components. We already valued Vue's size and flexibility and were somewhat familiar with it, so it was a natural choice. There's two main possibilities of handling your data when working with Vue: Either manage state in the components, or in a state machine, like Vuex.

Since many of the values need to be either changed or displayed in different components, keeping the state on a component level would tightly couple those components. This is exactly what is happening in the spreadsheet mentioned earlier. Fields from different parts of the sheet are referenced directly, making it hard to retrace the path of the data.

Keeping the state outside of the components and providing ways to update the state from any component decouples them. Not a single calculation needs to be done in an otherwise very view-related component. Any component can trigger an update, any component can read, but ultimately, the state machine decides what happens with the data.

Vue has a solution for that: Vuex. Vuex allows to decouple the state from components, moving them over to dedicated modules. Vue components can commit mutations to the state or dispatch actions that contain logic. For a clean setup, we went with Vuex.

Building the Vuex modules

The core functionality of the app can be boiled down to five steps:

1. Find the lot - Where do I want to build?
2. Define the building - How large is it? How many floors, etc.?
3. Further define any building parameters and choose a reference project - How many flats, parking lots, size of a flat?
4. Get the standards - What are the usual prices for flats and parking lots in this region?
5. Monetizing - What's the net yield of the building? How can it be influenced?

Those five steps essential boil down to four different topics:

1. The lot
2. The building with all its parameters
3. The reference project
4. The monetizing part

These topics can be treated as Vuex modules directly. An example for a basic module Lot would look like the the following:

// modules/Lot/index.js

export default {
  // Namespaced, so any mutations and actions can be accessed via `Lot/...`
  namespaced: true,

  // The actual state: All fields that the lot needs to know about
  state: {
    lotSize: 0.0,
    coefficientOfUtilization: 1.0,
    increasedUtilization: false,
    parkingReductionZone: 'U',
    // ...
  }
}

The fields within the state are some sort of interface: Those are the fields that can be altered via mutations or actions. They can be considered a "starting point" of all subsequent calculations.

Those subsequent calculations were implemented as getters within the same module, as long as they are still related to the Lot:

// modules/Lot/index.js

export default {
  namespaced: true,

  state: {
    lotSize: 0.0,
    coefficientOfUtilization: 1.0
  },

  // Getters - the subsequent calculations
  getters: {
    /**
     * Unit: m²
     * DE: Theoretisch realisierbare aGF
     * @param state
     * @return {number}
     */
    theoreticalRealizableCountableFloorArea: state => {
      return state.lotSize * state.coefficientOfUtilization
    },

    // ...
  }
}

And we're good to go. Mutations and actions are implemented in their respective store module too. This makes the part of the data actually changing more obvious.

Benefits and drawbacks

With this setup, we've achieved several things. First of all, we separated the data from the view, following the "separation of concerns" design principle. We also managed to group related fields and formulas together in a domain-driven way, thus making their location more predictable. All of the subsequent formulas are now also unit-testable. Testing their implementation within Vue components is harder as they are tightly coupled to the view. Thanks to the mutation history provided by the Vue dev tools, every change to the data is traceable. The overall state of the application also becomes exportable, allowing for an easier implementation of a "save & load" feature. Also, reactivity is kept as a core feature of the app - Vuex is fast enough to make any subsequent update of data virtually instant.

However, as with every architecture, there's also drawbacks. Mainly, by introducing Vuex, the application is getting more complex in general. Hooking the data to the components requires a lot of boilerplating - otherwise it's not clear which component is using which field. As all the store modules need similar methods (f.e. loading data or resetting the entire module) there's also a lot of boilerplating going on. Store modules are tightly coupled with each other by using fields and getters of basically all modules.

In conclusion, the benefits of this architecture outweigh the drawbacks. Having a state machine in this kind of application makes sense.

Takeaway thoughts

The journey from the coasters, to the spreadsheets, to a whiteboard, to an actual usable application was thrilling. The chosen architecture allowed us to keep a consistent set up, even with growing complexity of the calculations in the back. The app became more testable. The Vue components don't even care anymore about where the data is from, or what happens with changed fields. Separating the view and the model was a necessary decision to avoid a mess and tightly coupled components - the app stayed maintainable, which is important. After all, the students are using it all the time.

We were a bit surprised to find ourselves among “suits” (we were dressed casually). The talks reflected that: There was a lot of marketing. Companies seized the opportunity to showcase their soft- or hardware solutions.

Many speakers introduced us to Big Data . It was not so much fun to hear the same thing over and over again but at least we've got it hammered into our heads so much that we were beginning to think what it really means for everybody out there.

Big Data: The big picture

Big Data on one hand is a magic word to conjure up opportunities, publicity and Big Business. Frankly, we've had enough of the marketing “blah”.

On the other hand Big Data represents a shift in the way companies deal with data. For many years data has been piling up inside companies, but it was contained in silos. The idea that value could be added by correlating different data sources was not introduced until the big online players (think Google, Amazon etc.) were starting to gain insights – and make profits – by combining internal and external data.

It is true that there are opportunities for many companies, maybe especially for the bigger ones. “Big Data” consultants and solution providers take hold of the fear many companies might have about getting behind the competition if they don't jump onto the “Big Data” train.

But the concept behind Big Data (not the technology) is equally useful for any data. It doesn't have to be petabytes that you juggle around. Making better use of the available data is interesting for every company.

Why did many companies not get aware of this potential earlier? It has a lot to do with the traditional organizational structure. Collected data is usually tied to departments (IT, Finance, HR etc.) and these departments only rarely work together because they serve their individual purposes.

So the mission of the future is to break up these barriers and get the departments to share their data to help their business as a whole. How would that look like?

Small data

Say if you would not only store your e-mails but also bring them into correlation with your business. So your customer CompanyA is very important. You could do sentiment analysis of the e-mails and find out that with CompanyA the correspondence is mostly tense and unproductive, while the analysis of the communication with your CompanyB shows a positive atmosphere.

You could find out that you invest a lot more time into e-mails with CompanyA, but the return of investment is lower.

Gaining that sort of insight could help you to decide where you should invest in the future.

Or you could even detect sentiment changes in the e-mails from CompanyA while you are working on a project with them. So before a matter escalates you could already see a red light blinking on your dashboard and you can intervene before it gets ugly.

Or your email traffic analysis could indicate that you have not contacted CompanyC for two weeks and remind you that it would be time now to act upon that.

Small data has been coined to describe this kind of analysis:

“Small data connects people with timely, meaningful insights (derived from big data and/or “local” sources), organized and packaged – often visually – to be accessible, understandable, and actionable for everyday tasks.“

Small data also means:

“Know more, know earlier.”

There are almost no limits in that field. It is often not clear from the beginning what could be gained from the existing data. You have to ask new questions and look at your business in a different way than you might have done before.

Privacy

Because almost everything that happens in a company today is somehow tracked by some computer system you only need to know how to tap into these data streams and then can find out what is really going on instead of just relying on what people may want you to believe:

How long do the employees really work?
How does the social network inside the company look like?
Do some people write a lot of negative emails?
Which employees are likely to resign in the next 3 months?
Who is really helping your company and who is just pretending?
Are your employees criticizing the company or management more than last year?
What would be a good way to make sure you will get promoted?
In what way could a certain person be discredited so that people do not follow their advice?

Do these question make you uncomfortable? They should. All these questions could potentially be answered by combining and analyzing different data (e-mail, chat, coffee machines, copiers, fax, lunch cards, finance, cameras, HR, events, social media etc.)

You see the “problem” with this approach. And that's where we get into privacy concerns and ethics. First we may want to know what the Swiss privacy law rules. For our use-case probably the best source of information is called “Leitfaden über Internet- und E-Mailüberwachung” (not available in English) written by the Federal Data Protection and Information Commissioner (FDPIC).

From this document I quote section 7.2 (translated from German):

“As mentioned before, raw data (log files) have a direct reference to a person. For a person-based evaluation, direct references to individuals have to be prevented by assigning pseudonyms. For example, such an evaluation could answer the question: Are there employees in some department that send more than 100 emails per week? The employees that meet this criterion are to be listed with pseudonyms. This sort of evaluation can systematically be made without having to have any concrete suspicion of abuse.”

In the document this is called pseudonymous evaluation. It is permitted to answer all of these questions by the law and to use the results as one pleases. It is not permitted to use real-names in any such reports unless there is a concrete suspicion of abuse.

In the need to stay competitive, companies will eventually start to do this kind of analysis more and more. If you see companies doing it by using real-names, then you should not tolerate it but take the necessary steps to stop the practice. Personally I am not in favor of doing it with pseudonyms either.

Getting ready for action

Now if you think that Small data could help your company to get better at what you do, we suggest you to take the following steps:

1. Data inventory

The first step is to get an overview of your data. List the available data sources, possibly with short data samples that help with the assessment in the next step.

2. Data assessment

The next step is to assess the available data sources and identity correlations that could provide additional value for your business. You might be able to ask questions that you have never considered before because the data to answer them was not in sight.

Do not limit your assessment to internal data only, but also think about social media or other “public” data sources that might be related to your business.

3. Analytics

Once you have made up your mind up about what you can gain from what datasets and how it needs to be correlated, the data needs to be aggregated, prepared and analyzed. Depending on the field you are working in, you may need to look into Big Data technologies, involve data scientists (for cases with advanced analytics) or just aggregate and unify the data.

A possible milestone is a dashboard or visualisation that presents the insights from the analysis in an easily understandable way, preferably updated in real-time.

Summing it up

Big Data is not for everyone but the underlying concept that made it a hot topic can be very useful for any company that wants to be prepared for the future. We're looking forward to explore the topic both within our own company and with our clients.