Do you know the difference between scalability and performance? Which one do you need? Whether you are building, buying or selling web-based software, this post will teach you what you need to know in order to make the right choice.

I touched the surface of this topic in one of my previous posts, Scalable, Flexible and Cheap, Pick 2 you can’t have All 3, when discussing how to understand customer requirements regarding scalability. I learned that when a customer orders a software project from you and says “It must scale”, it can basically mean anything.

A few questions should help understand whether the customer means scalability from a technical- or business point of view. Business scalability means whether the business model can be scaled for a bigger audience. This is a bigger concept, covering market, process, localization and various other properties, including the technical scalability. Engineers often forget that scalability is not always only technical. However, in this post I will only discuss the technical one.

Having narrowed down to technical scalability, there is still room for misunderstanding. The customer may actually mean performance. If you are the subcontractor, you need to ask the right questions to figure this out. If you happen to be sitting on the other side of the table and you are outsourcing the software development to some other company, it is very good to know the difference too. There are plenty of subcontractors out there who just implement whatever was originally requested.

In various discussions, I hear all the time people say “this does not scale” when something is not as fast as expected or “Wow, this is really scalable” when something is fast. This is another reason for writing this post. Scalability and performance are not synonyms.

A high performance website is what one could call a “fast” website. Sometimes this is all you need. You might know in advance how many users are going to use it but require that the page responds quickly. This is usually achieved by code optimizations, bundling/packing/minifying resources and doing various other configurations.

A high scalability website is a website, which performance-level can be maintained by adding more capacity when the load increases. Some websites need to scale for growing traffic, others for growing amount of data. Many need to scale for both. A website can be slow and perform badly even if it scales well.

Regarding scalability, there are two types of scalability: vertical and horizontal. Vertical scalability means that capacity is increased by upgrading the server(s) to more powerful ones. Horizontal scalability means that capacity is increased by adding more servers of the same kind to the system. Horizontal scalability is preferred, since costs will often grow somewhat linearly to capacity. When scaling vertically, costs will often grow exponentially to capacity, since hardware gets very expensive at the high-end.

In practice, many systems only scale horizontally to a certain point. This is usually because they consist of many components, of which some cannot be scaled horizontally. When these components become bottlenecks, they need to be scaled vertically, resulting in a non-linear cost structure.

As earlier said, a website can perform badly even if it scales well. For example a website having a response time of 2 seconds could be considered slow and performing badly. If this website can handle 1000 users with one server, 10000 with 10 servers and 100000 with 100 servers, still maintaining its performance level (2s response time), it would be considered a highly scalable website.

Scalability and performance can have dependencies, both with positive and negative impact on each other. Consider a website, which architecture is changed into a more scalable Service Oriented Architecture (SOA), resulting in many more layers and web services. Requests may have to pass several layers causing a performance drop. On the other hand, some performance optimization tricks (or hacks) may be hard to scale. However, performance optimizations will often save resources, which will let you handle much more traffic with the same servers; hence, you will not reach your bottlenecks as early.

So, back to the question, which one do you need?

If you expect traffic or amount of data to grow, you need an architecture that can be scaled to meet the future needs by investing in proportion to the need. Hence, you need a horizontally scalable website.

If you know the traffic and amount of data is going to be within some reasonable known boundaries, but expect the website to be fast, you need a high performance website, designed for the specified amount of load.

Whichever you go for, remember that premature optimization is the root of all evil.

This post is about one common mistake startups often do. We did it when we built the PiX’n’PaLs photosharing website and later kind of repeated it in our e-commerce solution (read the story here). The mistake I am talking about is implementing more features without fully understanding what all the implications are. There are always a lot of great ideas popping up, but you can probably only pursue 1% of them. Trying to do to much is a common cause of failure; thus, the saying “Startups don’t starve, they drown“ has become so popular. In order to help others avoid drowning, I decided to list 10 things for startups to think about before implementing another feature. These are based on my own experiences, others’ stories I have heard or read about and a lot of common sense.

Experienced product managers use the word NO often. When I was younger I didn’t get them. Now, having been an entrepreneur myself, I understand them perfectly. I have really started to like Steve Job’s famous quote “Innovation is saying no to 1000 things”. Focus is essential and when you have one good idea you always find thousands of other ideas that would be cool. It is so easy to fall into the trap of implementing ideas here and there.

It is vital to understand that there are huge hidden costs in implementing a new feature. A feature may be really cool and seem to take only a few days to implement. An inexperienced product manager may happily keep pushing this kind of features into production even before product-market fit is found. The feeling is good when progress is being made and there is a lot of positive feedback from others. If the product gets poisoned with various different features that don’t fit well together before product-market fit is found, the company will be in a “feature hell”. At this stage it will be hard to find product-market fit, since all changes are slow and require a lot of work.

Let’s look a the different reasons why features can be poison:

1. Every new feature implemented requires more effort (the obvious one)

Every new feature obviously needs implementation effort. Additionally, a lot of documentation, manuals, FAQs, legal documents, etc. need to be updated. Some of these documents can be seen as features as well. Do you really need an FAQ or a manual? Will they become liabilities and slow you down? Furthermore, startups often put a lot of effort into analytics, which is used to learn about customers. This means that usage of the new feature need to be tracked. Finally, there may be a need to monitor the health of the feature in some cases, which will also result in more implementation work. What the developers initially thought would be a one day task suddenly became a huge operation that requires several people to get involved.

2. Every new feature may need maintenance and support

Having a new feature requires more maintenance work; someone needs to analyze all measurements collected, monitor the health of the system, etc. The new feature may also require more sales- and customer support.

3. Every new feature uses resources

New features may use more memory, hard disk or some other resources. All these have a cost.

4. Every new feature increases complexity

New features will increase the complexity of the system. This is dangerous for a startup still in search for product-market fit. Very often the complexity grows exponentially, because features have dependencies. This means that when you specify (or implement) feature X, you need to take into account that features A, B, C, etc. may or may not be used together with feature X. Sometimes it is possible to design everything in such a way that they are independent. But often when we solve real world problems that real customers have, we may not have that luxury. A well thought through spec and good software design will help deal with the complexity, but only to some degree.

5. Every new feature will cause some loss of flexibility

When there are more features and more complexity (indirectly), it will obviously be harder to do changes. Both technical- and business/logical complexity will require that a lot of time is spent on analyzing the impact of a change or new feature. All combinations must be thought through. This is very bad for a startup that has not yet found product-market fit and may still have to pivot.

6. Every new feature increases risk of bugs

Complexity will make it harder for developers to understand what the impact of a change is going to be. Also, high logical complexity will often also result in underspecification of new features and everybody will not share the same understanding of how the new feature should work together with everything old. Misunderstandings will lead to bugs.

7. Every new feature may introduce security holes

As complexity can lead to bugs and low quality of code, these can in their turn lead to security holes. Like with bugs, if the complexity is high and it is unclear how something is supposed to work, you will easily end up with some security holes in both code and application logic.

8. Every new feature must scale

The more features you have, the more potential bottlenecks you have. Complexity and dependencies between features are going to make it even harder to scale. Consider a consumer web application, which hits product/market fit and starts to grow rapidly. You have to cache content shown to end users. You have a requirement that the data must be fresh (changes visible immediately). The more different features there are that can indirectly have an impact on the data that is going to be shown to end users, the harder it is going to be.

9. Every new feature results in a longer learning curve

The more features and complexity there are, the longer it will take for new team members to be able to work independently and become productive. This applies to both developers and other employees.

10. Every new feature which is not used by the user will decrease usability and value

Every feature that is not used by the user, will distract the user from achieving his/her goal. Many studies show that a lot of options may result in the customer not choosing any because making the decision becomes so hard.

Additionally, when new features are added that do not perfectly fit into the software or are not fully thought through, the overall experienced completeness of the product will decrease. This happened to us in PiX’n’PaLs. If you thought our first version felt 50% complete, after we had added some more features that were not really core, you probably felt it was 40% complete.

At the time we realized that we should have been spending 80% of our time fixing old and 20% developing new, we had already implemented so many features that raising the degree of completeness required a lot more effort.

What happened in PiX’n’PaLs was actually that new features introduced new requirements. First our puzzle was missing a few pieces. Instead of looking for these pieces, we started building new puzzles next to the core puzzle, so the result was that we were missing even more pieces to the puzzle than when we started.

A startup should build a Minimum Viable Product (MVP). The product is either viable (can be used to validate a hypothesis) or it is not viable. The product cannot become any more viable by adding more features.

What to do then?

A startup needs to try out things and they may be complex, since customers need real world problems to be solved and the real world is complex.

Here’s some advice:

• Seriously try to keep things simple before product-market fit is found. Look for different trade-offs to reduce dependencies between features if possible.
• Narrow down the target customer segment.
• Focus on only one revenue stream, one Growth Engine, one X, one Y, etc. at a time. And do it well. Fail fast and move to next one.
• Avoid automating everything. Experiment by doing manual work.
• Really force yourself to think 80/20.
• Remove features and old stuff that do not add enough value.
• Focus on the goal and how to get there. Is this new feature really the way to get to the goal? There is most likely one biggest problem at the moment that you must solve in order to succeed. That problem is so hard and frustrating that it feels good to concentrate on some other tasks instead, like implementing some other new cool features. Don’t fall into this trap!
• If possible, keep things modular and reduce dependencies between features
• If you can identify key components that are unlikely to change during pivots, go for a Service Oriented Architecture (SOA). This is easier said than done ;)
• If you find yourself in the “Feature Hell”, try a Zoom in pivot, which means detecting one feature that is used, and making that the product and hiding everything else. If that is too hard to do, due to complexity, take the knife, be really aggressive and cut off all dead meat (features) from the product. This is what I would do if I would continue PiX’n’PaLs or any of my other old projects.

Picture Source: Jesse Sneed / Flickr (Creative Commons)

All over the web you find debates whether early stage startups should do Test-Driven Development (TDD). TDD is a development process where developers first write a test before writing any code. When the test is there, they first run the test to prove that it fails when the functionality is not there yet. After that they start implementing the functionality and continue until the test passes. TDD should result in higher quality and more maintainable code. TDD takes more time in the beginning, compared to just writing code directly. But when the complexity increases and there are no tests, things start to break and the velocity of the team will drop. However, with TDD the team will keep going with a somewhat more constant velocity for a much longer time. This is illustrated in the figure below with the green and orange curves.

There are plenty of people who feel that a startup should not do TDD, because it slows down the development in the beginning. The only goal of a startup is to find product-market fit as quickly as possible. Probably 90% of the code is anyway going to trash in the beginning. Progress is measured by validated hypotheses and not features, lines of code or anything else. The usual argument is that things change too quickly and the built MVP is anyway so simple that the benefit from TDD will not show off, since the code will be in trash before the velocity starts to drop. The blue curve in the diagram below shows theoretically the progress made when all code is thrown away after each experiment. One will have the opportunity to start fresh each time with no technical debt, so the average velocity will be quite high. This can probably only be done for the first experiments, after which there will be a better understanding of the users/customers and code reuse will start to pay off.

Some TDD fanatics argue that TDD actually speeds up development no matter what. The developers just have to learn to do it efficiently. Additionally, maintainability of the code is considered important in a chaotic environment where things change quickly. A final argument is that once product-market fit is found, it will be hard to take things forward and scale up when there is an enormous technical debt. This holds true for most software. The only exception would be apps, where the market risk is so high that one needs to start validating hypotheses with some really simple tests that go to trash immediately like illustrated with the blue curve in the diagram above. Completely new concepts for new markets meet these criteria.

Another common argument is that the software can be a bit buggy as long as it is good for validating a hypothesis. I read an interesting comment in Hacker News from a user called DanielBMarkham, that “technical debt can never exceed the economic value of your code, which in a startup is extremely likely to be zero”. I think this should be understood so that the cost of a rewrite is very low in a startup. This should not be be mixed with quality. Low quality with bugs will be a lot more costly than what was invested in writing the software. Even if the software is just a simple test of a business hypothesis, a bug may result in wrong measurements, which will eventually result in wrong business decisions.

How high quality do you need in your MVP? Let’s first look at what quality means. Colin Kloes wrote a good blog post about the importance of a team’s shared understanding of quality. He said “Software quality is characterised by how well the software has solved the user’s problem.”. Now think about Steve Blank’s definition of a startup: “A startup is a temporary organization designed to search for a repeatable and scalable business model”. The only purpose of the software is to help the founders find a repeatable and scalable business model. The software itself is the test; testing the business model that the founders have come up with. This means that quality is defined by how well the software is able to help the founders test their business model hypotheses.

In order to be able to test hypotheses, it is vital that all analytics are correct. A bug in the tracking of some important event that we are testing will result in wrong measurements, which may result in the wrong decisions taken by the founders. Also if there is a bug making the software unusable, the founders will never know if users were not using it because of a bug or because they were not interested. By putting it this way, quality is suddenly more important than ever.

Usually MVPs do not have much complex code, mostly just getters and setters. On the web they may sometimes be mostly html/css with some data retrieved from the database. In these cases there will not be a need for many unit tests. However, it is still a very good idea to have the TDD attitude from start. You do not know in which direction the MVP is going to evolve. You might want to have a couple of automated acceptance tests around the most important features to be sure they work as supposed to, ensuring your experiments are conducted correctly. Also, by doing this you will have the necessary infrastructure already in place to scale up operations when product-market fit is found.

In many cases the first MVP can be just an almost static HTML-page as an experiment to see how many users would sign up for a new concept. Some concepts can also be tested with something built on a CMS or even just static HTML-pages updated manually by a user. In these cases it would not make any sense to do TDD. However, I think these are special cases and at some point the founders are going to need own code to validate hypotheses further.

The software a startup is building, no matter whether it is an MVP or not, will have a purpose. If the software fails to deliver what it is meant to do, it is useless. Thus, bad quality is not acceptable. My conclusion is that the software is an important tool for the founders to validate their hypotheses. If the tool breaks or the technical debt becomes so high that it is no longer possible to use it for validating hypotheses with quick experiments, the founders will not be able to do their job of finding a repeatable and scalable business model.

Do you want to build a web app? You probably want it to scale, since it will become the next facebook and needs to be able to handle millions of simultaneous users. But you need flexibility in the architecture, since you may want to do bigger changes or pivot easily when needed. Naturally, you have a very low budget and the software needs to be built quickly. Sounds familiar, doesn’t it? These 3 things are important; however, you cannot get them all at once. The real question to ask yourself is which ones do you need at the moment?

All too often non-technical people put too much weight on scalability at a too early stage. When working as a consultant I discussed many projects with different startups wanting to outsource the development of their software. Many of them asked me for an offer of a web based app containing lots of different features. Sometimes there was a requirement that the app must scale to millions of simultaneous users or something similar to that. Everything was planned with a “Build it and they will come” mentality, which is very rare to work on the web. The problem was that most of these startups did not fully understand what scalability meant. People think of scalability in 3 different ways:

1. That the software is built in such a way that costs grow linearly when traffic grows or amount of data grows; meaning that scaling up happens by just adding more servers. This is referred to as horizontal scalability.
2. That the website can handle heavy traffic without any actions. This means that option 1) is done and there are lots of servers in place already.
3. The website is fast and responds quickly. I like to call this a high performance website, which I see as a different thing than a high scalability website.
   Unfortunately most people want type 2 scalability. Usually after learning about server costs and what needs to be done in practice, they want type 1 instead. But depending on the complexity of the app, type 1 may also require a significant amount of work in order to scale horizontally above a certain threshold.

A common misconception is that scalability is always good to have in place. The truth is that premature optimization in terms of scalability is very bad. Scalability often results in loss of flexibility. I define flexibility as the possibility to easily make modifications to the software or pivot quickly. This is a very important tradeoff that one needs to understand; therefore, I am going to explain what needs to be done from an implementation point of view to achieve these properties in web software. Jump directly to “What should you focus on”-subsection if you feel it gets too technical.

Flexibility

To have a fully flexible system, you may want to go with a relational database and a fully normalized schema. This makes sure there are no inconsistencies in the data whatever you do; Thus, it should be quite easy to implement any new functionality with various ad hoc queries to the database. You probably want to let the database take care of as much as possible. Preferably you want to run everything on one server, but having a separate database server and a few identical web servers behind a load balancer should not harm the flexibility.

Scalability

Depending on your application, your first step to a more scalable architecture may be caching. You want to minimize the load of the database, so you cache data that is used often. If you need to show fresh data to your users, you need to delete the affected cache entries when data changes. As you see, developers need to take this into account and they can no longer just do ad hoc queries to the database.

Additionally, you may need to take in new technologies, like a search engine. Whenever data changes in your database, you need to update the data in the search index and other potential storages to avoid inconsistencies. New features may also require schema changes, which may need to be taken into account in all different systems. You have lost a lot of flexibility.

Furthermore, to be able to scale for really big datasets, you cannot just do complex database queries. You need the data to already be in the format you are going to use it. This requires denormalization, which means that you have a lot of redundant data. An example of denormalization could be a photosharing website keeping a separate count for the number of comments an image has, in order to be able to easily sort images by number of comments. Without this number, the database would have to count the comments each time. The more the database is denormalized, the harder it is to do quick changes. You may also want to switch to a NoSQL database if you plan to denormalize a lot.

Finally, scalable apps usually use plenty of application specific tricks. These tricks may not work if there are bigger changes to the application logic. This means that when functionality changes, the developer needs to come up with a new way to scale; thereby spending a lot more time implementing the change.

Flexibility and scalability together

Sometimes you need both flexibility and scalability. In this case you have to define within what scope you need to be flexible. Full flexibility is not going to be possible. However, by planning how to migrate data, re-calculate redundant data and using several other tricks, it should be possible to be flexible within the scope.

You may be able go into a Service Oriented Architecture (SOA) and build web services for retrieving any kind of data needed for features within the predefined scope. This way the scalability can be implemented in the web services (application servers) and developers can have quite good flexibility when developing on the web servers. The broader the scope is, the more expensive it is going to be.

What should you focus on?

Now we know that scalability will require certain design decisions that will make it harder to do changes in the future. If there is a need to be both scalable and flexible, you need to have a good understanding within what scope to be flexible. The cost wll be higher since the implementation is a lot harder. I think the best way to illustrate this equation of flexibility, scalability and cost is by using a ternary plot. Cost would refer to both time and money:

Now that you know the basics, the question is where do you want to be on the triangle? The wrong decision can be fatal for a startup, so understanding the pros and cons is important. I think it all comes down to what kind of a startup you are and at what stage you are.

Most new web apps built by various startups have enormous market risk, especially the ones trying out completely new concepts in new markets. Every startup has a runway of a certain length, during which it needs to find a profitable and scalable business model and fly or run out of cash and die. It needs to test and validate hypotheses as quickly as possible. This needs to be supported technically, therefore it is extremely important to be as flexible as possible in the beginning.

When product-market fit is reached, usage starts to grow rapidly. Now scalability becomes more and more important. It may be tempting to still keep things flexible and go for various other great ideas that come up here and there; however, this increases the cost and can be very dangerous for a startup which does not yet have all its processes in place. Therefore, the best approach is probably to productize what works and just scale up loosing as much flexibility as necessary. The fewer features got implemented during the flexibility stage, the easier it will be to scale. Due to this, it makes sense to keep the feature set as small as possible during the flexibility stage by removing unused or otherwise not working features.

There may of course come a day when there is a need for flexibility again, for example when the product is going to enter the mainstream market. The mainstream market usually requires a whole product, which naturally is going to require a lot of effort to build. But then again at this stage, the company has probably generated some revenue and raised more capital and the processes and technical competence are at a much higher level.

The point I am trying to make is that it is extremely important that everyone making product decisions understand what the cost of scalability is. It is so easy to fool yourself that whatever you are building is going to hit the jackpot and build too much scalability into the system. When you learn that it didn’t work, you are going to need to pivot. Either you pivot keeping the scalability but with high cost or you drop the scalability. Jumping back and forth in the triangle is going to be costly and is probably going to result in a lot of technical debt.

I would be very glad to hear what different successful and/or unsuccessful startups have done regarding flexibility and scalability at the early stages of a company. Any experiences?

I have read a lot of startup books and books about software business in general. Most of them describe different activities, that software companies should be doing with the help of various tools, frameworks and processes. As a reader I used to feel overwhelmed with all the information, especially when every idea was presented as something important that cannot be neglected by any company under any circumstances. I lacked perspective of which activities should be prioritized and which should be completely skipped at a certain stage of a company.

Most of the books presenting great frameworks for analyzing different things, will not tell you how to obtain the input needed for these frameworks. Bad input leads automatically to bad output, which means that these frameworks are completely useless unless you have all the data. Bigger companies may already have a lot of market data and can somewhat easily get more, since they probably already have a good understanding of the market. Startups don’t have. Many books let you believe that you either need to know all this in advance or if you are lucky they present a way to do market research in some classical way that only a big company can afford to do well.

I came across 3 books that managed to put everything in its right place for me. These books helped me to understand what is really needed in a startup and when to focus on it.

1. The Four Steps to the Epiphany: Successful Strategies for Products that Win, by Steve Blank

I found Steve Blank’s blog a couple of years ago from a TechCrunch article about the long lost formula of startup success. After reading some blog posts about the Customer Development model I realized that this is exactly the missing piece to the puzzle, so I bough the book. It’s a step-by-step guide to build a software company. It tackles the core problem of how to find which customer problem to solve and how, using an iterative process called Customer Development. All the other business books that I had previously read had made me think that good entrepreneurs know from the beginning which problem to solve. Also various books about usability, focusing on finding the solution to a predefined problem by doing interviews, usability tests and various other activities with the customers had as well put me off the track. They made me blind to understand that the real point is not to directly find the solution to the customer problem but instead to find a problem that is worth to solve; a problem around which it is possible to build a profitable and scalable business.

Order you copy here!

2. Crossing the Chasm: Marketing and Selling Disruptive Products to Mainstream Customers, by Geoffrey A. Moore

Crossing the Chasm has been said to be the bible of entrepreneurial marketing. It discusses the technology adoption life cycle and the so called chasm, which lies between early adopters of a product and the early majority. It is known to be very hard to get a technology product from visionary customers into the mainstream market. This book is most certainly a good read for anyone who is going to get their software product out in the mainstream. One way to look at it, is that Steve Blank’s book is the one helping you to get to the chasm and this book helps you cross it. But I think there is much more to it than that.

For me the book additionally helped me put all the different activities I previously had learned from other books into the right stages in the technology adoption life cycle. I had read from various other books several statements, such as “You must create a strong brand for your company”, “You need to prove that you are the market leader”, “There must be something new that gives a competitive advantage to the first customers”, etc. All these statements are true, but at different stages in the technology adoption life cycle. This book helped me understand that. Also understanding the behaviour of customers from different stages helps a startup understand which customers to focus on and which to leave for later. In my previous company, with PiX’n’PaLs StoreFront, I wasted an enormous amount of time trying to sell to late majority or laggards.

Order your copy here!

3. The Lean Startup, by Eric Ries

Although I had been reading Eric Ries blog before the book and was already quite familiar with many of the ideas in The Lean Startup book, I found it to be very useful in helping me understand how to validate business hypotheses with an agile software development process; it is actually possible to take it so far that all the “stories” that developers build are considered tests. Instead of an engineer getting a job to implement a feature, he would get a job to find out whether a feature helps the company in its goal. This goal may be to grow virally, convert users to paying customers or something else. Running a process like this will get a product built with only features that are needed and the chances of reaching product-market fit are higher. I think it contained many of the missing pieces to the puzzle regarding how to actually run the tests and what to measure and what not.

Order your copy** here!**

Nowadays, most of these theories in these three startup books may be more known in the startup community than they were when I was an entrepreneur. Still, if you have heard pieces of advice from different people here and there, you may have a poor understanding of the big picture. Thus, I recommend that anyone who intends to start a software company the first time reads these books.

I recently had a conversation with a friend, who also has a programming background but has moved more into a managerial position like me. We talked about when programming is fun and when it is not. We have both had moments when there is nothing else in the world but the one thing you are working on 24/7. You have so much adrenaline in your blood that you couldn’t sleep if you wanted to, so you just keep on hacking the whole night. These are the peaks that make software development the most interesting profession in the world. This of course requires the right circumstances.

I think it all comes down to how much “magic” there is in the stuff that you implement. It may be something really Hi-Tech to you, that you perceived as magic at some point in your life. Or it may also be something really simple but has the potential to spread virally to millions of users. However, there must be some magic in it to be able to unlock that kind of a productivity boost. My biggest moment of magic was when I built my first chess engine in 2002.

From the day I played chess against a computer the first time, long before I took up chess for real, I had been wondering how it is possible that the computer can think. When I got a little bit older I understood that it has to search through the moves tree and check what results in the best position. But still, without proper understanding of algorithms and data structures, it felt to me like some kind of magic.

Before directly going to the story, I’ll give a quick background to how I got into chess and writing a chess engine. I have always been a fan of strategy games, ever since I got my first computer and played Civilization 1, Sim City and later C&C and similar games. However, as much as I would have wanted to become very good at any of them, I knew that the knowledge would become obsolete one day when a newer game was released. That put my view of chess into a completely different perspective, since that game is going to stay unchanged forever.

My cousin had taught me to play chess since I was small, but I never really enjoyed it at that age. But when I was studying for the first year in high school, surprisingly chess became very cool. We were about 15-16 years old. People were playing during the breaks and even girls played or watched. I am not talking about nerds and geeks. This was an all-embracing phenomenon. I immediately felt that this is my game and I must be best at it. I started playing a lot. I read a few chess books, joined a local club and after a while I played in tournaments.

When I took a beginners course in Java programming at TKK in spring 2002, the course had a bigger project assignment at the end with several different topics. One topic was to build a chess game, but it was without possibility to play against the computer. I couldn’t let the opportunity pass, so I just had to choose that assignment and also go for the artificial intelligence as extra. Students were allowed to do changes to the scope if they were reasonable.

This was a very interesting project. I still remember well the moment when I got to try the AI functionality for the first time. I had already implemented all the functionality necessary to make a list of all the legal moves in a position and to make moves on the board. There was only textual representation of the board at this time, but that was all that was needed. I was at a computer class in TKK, where I used to work to be able to concentrate. It was 5 am already and I had fully lost my sense of time.

I used the negamax algorithm, one of the simplest forms of search. I had implemented with a very simple static position evaluation function, only counting the score of the pieces at the board. When the first bugs had been fixed so that it did not crash immediately, it actually played. It was able to search to a certain depth and suggest a move that proved that it had some kind of intelligence. It was able to think. I immediately jumped up and said to myself: “It’s alive, it’s alive!”. Too bad there was nobody else listening.

These kinds of peeks came every now and then when I was improving the AI. Adding alpha-beta search and other improvements to the algorithm allowed the engine to search deeper and do more complex tactical combinations. Improving the evaluation function to take into account positional advantages, such as positions of pieces, king safety and pawn structure made it play a lot more like humans. I still remember the Wow-feeling when I realized it was thinking strategically and it was playing quite coordinated attacks against me.

When the school project was over, I decided to continue the development as a hobby. I rewrote the engine in C, while doing several other improvements as well. I was doing this for a year or so. My engine (Hoplite) implemented the Chess Engine Communication Protocol, so it was able to play against other engines. Hoplite was playing in several tournaments around the world. It was free and open source, so tournament organizers could easily enter it into a tournament.

I had a lot of fun watching how it performed in a tournament after I had done a new release. Also during development I had set up my own tournaments, where I had several engines playing against each other. I usually had 2-3 variants of my own engine and then a couple of other engines for reference. Tournaments were automatically played with a script, so I could have it running during the night and check the results later. If a new feature showed good results, I added it to the mainline. I almost took this process so far that I would have used evolutionary algorithms to get optimal configurations. This would have been overkill, since the biggest improvements would have come from completely different optimizations.

The last time I touched Hoplite was probably in 2005. There came other more important things and at those days I had already realized that making it play really well would require rewriting some parts and a lot of time, which I unfortunately did not have. Maybe one day I’ll give it a shot again.

These are the peaks that make programming so damn interesting. You need to implement stuff that has magic in it. Every time you do that, you raise the bar and it gets harder to find any problem with magic again. During the last 10 years I have been raising my bar all the time and nowadays it is getting very hard to find something that I could implement with as much magic in it for me as the chess engine once had.

This raises an interesting question: What can a software company do to let all developers experience these kind of moments? Should a company give a lot more freedom to developers to work on things they like even though they are not 1st priority? It’s always a tradeoff and it depends on what kind of software you are building. If there is development of complex algorithms or growth hacking involved, it may make sense to let a team operate outside the usual process. Product management can define the boundaries within which the team can make decisions themselves and then give green light to do whatever they want within those boundaries.

For those interested, I created a simple page for Hoplite, from which the source code or Windows binary can be downloaded. The original Hoplite page was removed from TKKs servers when I graduated. Follow this link: Hoplite Chess Engine

Many web apps rely fully on viral spread. In order for this to work, a virality coefficient higher than 1 is needed; meaning one user invites more than one other user into the service. Lots of well known bloggers and authors, like Eric Ries, state that you should postpone your full launch and iterate until you have reached a virality coefficient above 1. I believed this for a long time and still think it makes perfect sense in many situations. However, I think there are exceptions.

The usual process to create a viral app used to be to start with a quiet launch to be able to optimize the viral loop and doing a full launch when reaching something above 1. I think optimizing the viral loop this way is still a must. But it is very hard to emulate real conditions. There are many variables that will have an impact. Some can be emulated, others are harder. For example, the conversion ratio of invitations to signups will most certainly be affected by how well known the brand is. If a user gets an invitation to a well known app like Angry Birds, the user will of course be more likely to sign up and try it out, than any previously unknown app.

My hypothesis is that there are many situations, where a virality coefficient of less than one during tests will be sufficient in the real world. The reason is that it may be possible to improve it with some big PR event, where you influence the masses before they get reached by the virally spreading invitations. I look at this like in war, where you first fire with the artillery (PR) and then send in the infantry (invitations). Also in many cases there may be economic externalities, and reaching a critical mass will also have an impact on the virality coefficient. Therefore, the actual post-launch viral coefficient will be above one, which is all that matters.

Even though the ecosystem of web apps is unlikely a “zero sum game”, someone’s gain in virality is often away from someone else’s. This happens when everyone is optimizing ther virality in every possible way. Apps are fighting for eyeballs everywhere in feeds, emails, etc. The more there are players using the right methods, the harder it will be to reach a coefficient above 1. Therefore, one could assume that some apps would have to do a big PR launch just to get over the edge. I would be glad to hear what others think and what the new startups that recently launched a successful viral app did.

This is the final part of The Disruptive Adventure, where we see that the whole adventure led to something good, even though it didn’t go as initially planned. This post makes more sense if you have read part1 and part2 first.

From the beginning we had all the time planned to sell photo books in PiX’n’PaLs. At this point we had built the software needed to setup a store selling customized photo products. Additionally, we had sold the software to customers, who had all the capabilities to print photo books and other products. Customers of StoreFront were more than happy to handle our production. This was also technically easy for us. All we had to do was transfer our own orders to our production partner’s account. We could charge our end users whatever we wanted for their products and then pay the production partner an agreed amount.

We thought that others would also be interested in such a feature for their website, that made it possible to sell print products from all their pictures. Therefore, we decided to build something more generic, that everyone else could use as well. The new service was called PiX’n’PaLs PhotoProducts, which helped publishers monetize their website by selling print products from images. We took care of payment, shipping and everything else. The publisher got paid a 20% commission.

The service had two main features:

1. A publisher could add our code snippet to their website, making it possible for users to buy print products directly from an image. Clicking on the buy link opened a small shop, where the user could select which photo product to order from the image and use basic image editing tools, such as cropping, rotating and resizing the image.
2. Integrate our whole e-commerce solution to their website. When a user ordered print products, they had all the photos immediately available. This was practical in PiX’n’PaLs, where users had access to all their pictures and their friends’ pictures from the shared events. The events appeared as directories in the shop.

We launched with one of the biggest Finnish websites, Riemurasia, which enabled this feature for all its photos. The launch looked really promising and we immediately got lots of orders. Unfortunately our luck ended pretty soon when our feature had to be limited to only certain photos, due to potential copyright issues. Later it also turned out to be so that many orders got canceled, so in the long run it wasn’t that easy money after all. We would have needed many more big websites and better products with higher margins. Finland was definitely too small.

Launching this globally would have required lots of funding in order to get production and logistics to work well internationally. Getting deals with big international companies was hard and plugging into their production pipeline turned out to be technically impossible, since they did not have any APIs. It’s hard to tell whether this would have worked, since we were running out of money and had to leave it there. Potential investors were not too crazy about investing in a declining market either. The numbers did not fully work out.

The safe way to fail for many entrepreneurs is to start doing consultancy. This is what we did. We started doing projects for others to get some cash quickly. This seemed more comfortable after a long adventure with very high risks, so we kept doing it for full time until we joined one of our customers (Kiosked) as partners. We had already built the first version of their software and had all knowledge to take it forward from there. Today I am working there as the Chief Technology Officer and living really exciting times now that the company is growing rapidly.

PiX’n’PaLs is still running and used regularly by a lot of users and PiX’n’PaLs StoreFront is still used by two print-labs. These products are not developed further anymore. Some maintenance work is done every now and then to keep paying customers happy. If we would want to continue one day, we would have to update the UI and graphics completely, since they were optimized for IE6 and other ancient browsers. We have shut down the PhotoProducts and our own store (EasyKuva) which was one service built on the PhotoProducts API.

Well, you probably wonder: was it worth it? Absolutely! I don’t regret anything. The things I experienced and learned could not have come from any normal nine-to-five job. And as it turned out to be, one adventure led to another.

_This post makes more sense if you have read part 1. This post is about the 2nd phase in my company’s life.
_

We had been occupied with our real jobs for over half a year, only doing small fixes to PiX’n’PaLs in the evenings or weekends. I was working at a lab in Helsinki University of Technology, doing research and writing my Master’s Thesis. When I graduated, it did not make much sense for me to try to extend my employment agreement any further, since I didn’t have plans to continue with post graduate studies anyway. Jarl was also occupied in a similar position in his school, doing research in a different field. We had a strong belief that we could make PiX’n’PaLs work if we made a few improvements, so we decided to give it another shot.

The problem was that few new users saw any immediate value in using PiX’n’PaLs. For a user to see any value in PiX’n’PaLs, there had to be many events in the user’s story with many pictures taken by different people. A lot of effort was required for a user to get to that point, therefore it was easier for them to just post photos on Facebook. We felt that there had to be some bigger reward for going through all this. Nowadays I understand that end users on the web think really short term. There must be instant gratification for every small action.

Instead, we chose something more long term. We felt that adding a possibility to buy a photo book from an event would make it worthwhile for users to go through all the extra effort required. We knew that people usually want to create photo books or albums from events like weddings, bachelor parties, etc. Very often it required getting everyone’s pictures together into one place before making the book. This was often a hassle when everyone put their pictures to different places or sent them by email. We thought that PiX’n’PaLs could solve this problem.

This new feature required building an e-commerce solution with photo editing capabilities, since there were no solutions available that would have fitted this purpose. There were some solutions that could have been used for selling t-shirts and other simple products, but nothing for a photobook that we could have integrated into PiX’n’PaLs.

During this time we got into a pre-incubation service in Technopolis Ventures, where we got some help from an external advisor. Together we decided that it does not make any sense to simply build this feature only for ourselves. Instead, we decided to look for someone else who we could sell it to, thereby financing at least part of our product development.

After doing a few hours of research, we learned quickly that the ones who could potentially be interested in buying it were either small print labs and photo shops that needed an online shop with a photo book editor or then a few bigger players that had their own web shop already, but could potentially be interested in a high-end photo book editor. We tried for a while to sell this to one of the bigger players as a project but didn’t get the deal. Additionally there were photographers that could sell print products of photos they had taken. Both small print-labs and photographers had small budgets, so building something for them would not make sense unless it was possible to sell to enough customers.

We actually got on the right track and came up with a MVP to suggest to them. The MVP was a simple file uploader. Since so many small print labs did not have anything, we thought that a simple photo uploader would already give them something and would immediately bring money into the almost bankrupt company, and from there it would be easier to go forward.

I started cold calling all these companies and tried sell and find out if there were a need for this. I was not aware of Steve Blank’s Customer Development model at the time, but I had heard pieces and parts of similar theories from other entrepreneurs, mainly about the importance of starting from selling instead of coding. What I didn’t know was that you really need to do this iteratively and pivot when the current idea does not work. I did not fully understand the importance of iterating to find the best problem to solve.

I think we did not understand that the real purpose of selling was validation (or invalidation) and had the “always be closing” mindset at a too early stage of the company. Later, I have been joking that during the validation process, if customers don’t buy, you should open a champagne bottle and celebrate that you found one way that didn’t work and are now a lot closer to success.

We met some of the companies and had long chats with them and learned a lot more about their business. Even though we iterated the product based on what we learned, all our iterations were to improve our solution to the same customer problem. We should have iterated on the problem first, to find which problem makes most sense to solve, from a business point of view.

Anyway, a simple photo uploader was not enough and everyone wanted lots of features like selling coffee cups, mouse pads, etc. Most of them were really interested in buying the product. Some that were already using the competitor’s product were thinking about switching if certain features, that the competitor’s product lacked, were implemented. It looked really promising. We were not able to sign any deals immediately, since everyone first wanted to see a demo or try it out a little bit before buying.

So the spec changed to a full e-commerce solution with photo product editors. We decided to brand the product “PiX’n’PaLs StoreFront”, since PiX’n’PaLs was what the customers recognized and remembered us by.

We started implementing the product with most of the needed features. We should have continued with our customer development and tried to find a MVP that the customer is willing to buy before it’s implemented. We should also have validated that the business model is profitable and scalable, which we did not do well enough. This was because we had left the price question a little bit open and talked about a commission based model, making it very hard to predict how much we were going to earn. We thought the commission based model made sense, since we did not have a good understanding of the volumes or the margins in that business at the time.

Afterwards it is easy to say, but best would have been to require them to sign a contract and agreeing to a constant monthly fee. Unless the product is really a must have, they would not have signed. This could have invalidated the whole idea (champagne!). If not, it would have meant that the idea is validated and has good chances of succeeding. The contract could have had a minimum feature set specified and a deadline, speeding up things. Customers would also be much more committed to help with development by giving good feedback if they already had put money in the project.

Iterations took longer than planned for us, because small print labs were often quite busy and did not have time to immediately have a look at the new version and give feedback. The owners had to work themselves in the store and usually they could only find once a week time for a quick chat over phone. When they finally had time to try out the new version, if they felt that it still needed improvement or they came up with a new feature that would be needed, the launch was likely to be postponed by at least a month. We kind of got into a “feature hell”, implementing too much customization to fit everyone’s need before the product was live anywhere.

It took a horribly long time to implement everything from custom photo product editors and photo book editors to shipping cost calculators, payment options, etc. We even made the shop as a widget, running seamlessly on the customer’s website. It was possible to customize the colors of all texts and elements and lots of other things as well. The customer could configure the service to sell almost any kind of print product having a photo on it, just by defining the dimensions and the position where the photo should be printed on the product.

The photo book was probably the one particular feature we spent most time on. It was highly customizable and the first version already contained lots of unnecessary features, such as infinite undo/redo and an intelligent autofill feature that laid out all the user’s images in a nice way on all the pages, depending on their image dimension, number of pages, etc.

All the different configuration options that we provided customers just made the adoption harder. Likewise, the seamless integration into the customer’s website required that there were at least one technical guy on the customer’s side. Otherwise we had to help embed it. Nevertheless, a lot of work was needed both before and after the customer had said yes. When we learned how much commission we were getting, we quickly understood that the business could not scale and be profitable if we were going to pay full salary to sales people and support engineers that would handle the whole process. Luckily, we had not hired these people yet.

Bigger photo shops could possibly have generated enough money, but they already had some solution that was tightly integrated to all their equipment and processes. This market was more mature and these customers would require a so called whole product, to which there exist a lot of additional equipment and other software that can be connected to it.

For small photo shops, there was also a lot of pressure coming from big international players. The online market was already taken by bigger players and even larger international players were trying to enter the market. Small businesses had no chance competing with prices, so instead they went after the only segment left, which were customers needing high customization, assistance in the store and quick delivery (you could often get your products within an hour if you were at the store). The small shops seemed to prefer customers coming into the store instead of buying from the web. This was also a shrinking market and more and more photo shops were closing their doors, so it was time to get out of there.

To be continued…

This post is about my own startup (Disruptive Media) and how we pivoted and where it took us. I have split it into 3 parts, which all describe a specific phase in my company’s life. I am not going to go too deeply into all the lessons learned, but instead focus on the story and leave the many lessons for coming posts.

In 2007 I founded Disruptive Media Ltd. together with my colleague Jarl Törnroos, who I know since the late 80’s when we went to school together. Both of us have had programming as a hobby and written 2 computer games together back in 1998; one poker game and one labyrinth game. However, after that we have not had any experience working together until we founded our company.

In 2006 I was playing around with lots of different ideas to found a company. One was an improvement of a private photosharing website Jarl had built a few years before as a hobby project. Jarl’s website was used by all his friends and it was kind of used to document the life of our group of friends. All albums were group albums, usually created for crazy parties and other events.

More and more people were asking if they could use Jarl’s website and the existing users seemed happy. Sadly, it was designed only for one private group of friends, where everyone could see all the albums, so it would not have scaled. Therefore, I proposed building an improved version of this, which would allow anyone to setup an own private gallery with group albums. The idea was not only to target groups of friends, but also sport teams and similar.

The idea grew in our minds for about half a year before we got started. I was still studying at TKK, which I had just continued after an over 2 year long adventure in the Finnish mobile game company Sumea, which during my time was acquired by Digital Chocolate. Jarl was also working, so there was little time for our own project, other than brainstorming ideas every now and then. At those days we had a third member in our team, who dropped off before summer, during which we had taken some time off to start hacking for real.

Our idea had changed a bit from the initial one. Our idea was now a service to document your life (or the story of your life) in a collaborative way with your friends. We had dropped the idea of groups, since there are no real boundaries between groups of friends. The idea was that any user can create events and invite their friends to these events, so that everyone can upload their photos to the same event. All these events a user belongs to, form the story of that user’s life.

In addition to this, we had an innovative friend/contact list, where you became friends with anyone you had been to an event with. We thought it was a killer idea to quickly build up a social network that so many companies had tried to do but failed.

Early usability tests and interviews had revealed that some users would like to publicly share images from events. Events and pictures in them were private only to members of that event. So, we came up with another “great invention” that users could publish any events or images from their story on their public profile. This way, users who never took any photos could also share their story with the world.

The name of the service became PiX’n’Pals, after several long discussions. We would not have wanted to brand it with anything so tightly linked to pictures, but all good names for story, events, etc. were taken. Finally, we decided that since the service is going to be mainly about pictures in the beginning anyway, we can as well just go with name PiX’n’PaLs. If we manage to get a proof of concept, we are probably going to seek funding and probably even relaunch with a new brand.

Our plan was to make money from selling print products, like photo books, from the photos. We also saw the photo book as a value adding feature that would attract even more users. Additionally, we were going to have ads and possibly premium accounts at some point.

We should have concentrated on the core idea and launched a Minimum Viable Product (MVP) to test the concept, as we initially had planned even though we never had heard of the concept of an MVP. We kept building more features, since we always felt that “the service needs X because Flickr has it too” or “he/she said he needs that feature”.

In addition to all the features, we also felt that we needed an FAQ, long help texts, take a tour, etc. All this took a lot of time to make, but that was not the worst of it. All the features and extra stuff turned out to be a liability later when we wanted to make changes, since they needed to be updated every time. We even hired a lawyer to write our terms of use and privacy policy for the first version. I will probably write another post on this topic soon, to list all the reasons why features are poison and so dangerous!

Time spent in “feature hell” was away from implementing other planned things, such as good tracking of user behavior and support to run A/B tests to optimize funnels. Our tracking was something quickly put together before launch, only measuring vanity metrics, not really showing whether we are making progress or not.

I had read a lot about A/B testing and optimizing conversions, but did not know that it is the most important thing in our business. Optimizing virality, by measuring how many users one other user invites (virality coefficient), is the path to success, unless there are other ways to acquire customers at a decent cost. Of course, this means that the goal of all product decisions regarding design, features, etc. should be to improve the coefficient. This must be verified with an A/B test to be sure that progress is made. When the virality coefficient is something above 1, the next thing to optimize would be revenue from a customer. These activities must take place after a quiet beta launch, but the processes and infrastructure should be in order to be able to conduct experiments.

Another area where we could have spent more time and money on was graphics (and UI & UX in general). In the beginning we were kind of on the right track with mockups and wireframes, even doing usability tests, but the time pressure made us cut corners and just get it out. More features required more graphics and more complex UI, which required more time from us. It usually takes a lot of time for an artist to do good looking graphics and engineers are rarely top artists. The worst possible combination is when an engineer draws graphics in a hurry.

It was time to launch. Little were we aware of facebook, which was getting more and more popular and also slowly opening up in Finland. One of our big mistakes was not to try out all the other services, to fully understand what the existing services were already capable of. We had read about facebook and probably signed up a user account, but couldn’t imagine that people would really start using it a few months after that.

We realized pretty soon that PiX’n’PaLs did not grow exponentially. In the beginning friends started to use it and they invited their friends and so on, but the branching factor of the tree became really low after a few nodes (These were people we did not know at all).

We started collecting feedback from users and pop out more features and improvements. Generally a good idea, however, all these improvements were targeted to existing users. Instead, all improvements should have been directed by measurements of the virality coefficient.

For new users the service was visually “empty” and there were not really much to do, unless you started to create events and upload photos. Additionally, the service had lost its focus when we had implemented more and more features, like messaging and stuff that was not around the core idea.

Measurements showed us it was growing linearly with more people coming in every now and then from different sources, like blogs writing about us, etc. These users created some viral spread, but with a vitality coefficient way below 1, cooling off pretty quickly.

When I look back at the post it feels like we did everything wrong. We did lot’s of mistakes that we learned a lot from, but we also did some things right. Lots of companies spent even more time trying to get everything perfect before launch and most of them burned a lot more cash. For us this was definitely not the end of the story. I’ll tell you what happened in the next post coming soon…