02.Development Philosophy

Testing over talk: Data driven

Product development cycle
Product development cycle

Pursuing data-driven development

Indeed pursues data-driven development. Instead of spending time discussing various ideas, testing them and measuring and analyzing all the results. We use that data to determine how to improve things as the next step and what to do in the following week. We swiftly repeat this cycle and quickly show users ideas to see if they like them. That eliminates wasted time on failed plans. Failing fast and then learning is vital for progressing to pursue greater value.

Frameworks behind data-driven development

Testing

A/B test everything, running 103 tests per month

Indeed conducts A/B tests for everything from user interfaces to back-end algorithms when changing applications and services. We run 103 tests in 315 variations per month. Indeed developed the “Proctor”(A/B testing framework) to efficiently implement and manage massive A/B tests. GitHub hosts this open source code, which many companies are using.

View Proctor GitHub

Measure

Accumulate 5 billion logs everyday

Data is vital to evaluate A/B test results. Indeed built the “logrepo” system to store more than 500 types of site logs. The number of log events reaches 5 billion per day. “logrepo” can store measured data to prevent it from becoming corrupted or lost by checking its consistency and confirming transmission before we analyze it. Thus, we can use massive yet high-quality data for analysis.

Analysis

Structure to swiftly and statistically analyze data

Decide next actions based on the analysis and evaluation of measured data. Indeed developed “Imhotep” analytics tool for quick decision-making for improvements. “Imhotep” can quickly provide the analytical results from various perspectives even with the change of detail conditions. This is all Web-based, so simply sharing the URL makes it possible to understand what queries produced what results. This setup makes it easier to deepen team discussion.

View Imhotep GitHub

Engineers enjoy agile development

Indeed releases software weekly or even more frequently by running the cycle above. Engineers prefer smaller and quicker development cycle than those are once or twice a year. That is because they can swiftly understand the results of their work and get excitement from amassing many successes. There are also fewer risks of failure, and even if it failed, the impact would be smaller. Thus engineers can work in less stressful environments.

The benefits for engineers of agile development
The benefits for engineers of agile development