Google is born

See, Yahoo's keyword search was easy to trick. To get your page ranked highly, you could just repeat keywords hundreds of times, hidden with white text on a white background. - One thing they didn't have in those early days was a notion of quality of the result. So they had a notion of relevance saying, does this document talk about the thing that you're interested in? But there wasn't really a notion of which ones are better.

- What they really needed was a way to rank pages by both relevance and quality. But how do you measure the quality of a webpage? Well, to understand that, we need to borrow an idea from libraries. - So I'm old enough that library books used to have a paper card in it that was a stamp of all the due dates of when it was due back. You took a book and if it had a lot of those, you said, "Oh, this is probably a good book."

And if it didn't have any, you said, "Well, maybe this isn't the best book." - Stamps acted like endorsements. The more stamps, the better the book must be. And the same idea can be applied to the web. Over at Stanford, two PhD students, Sergey Brin and Larry Page, were working on this exact problem. Brin and Page realized that each link to a page can be thought of as an endorsement. And the more links a page sends out, the less valuable each vote becomes.

So what they realized is that we can model the web as a Markov chain. - [Derek] To see how this works, imagine a toy internet with just four webpages. Call them Amy, Ben, Chris, and Dan. These are our states. Typically, one webpage links to others, allowing you to move between them. These are our transitions. In this setup, Amy only links to Ben, so there's a 100% chance of going from Amy to Ben. Ben links to Amy, Chris, and Dan, so there's a 33% chance of going to any of those pages, and we can fill out the other transition probabilities in the same way.

So now we can run this Markov chain and see what happens. Imagine you're a surfer on this web. You start on a random page, say, Amy, and you keep running the machine and keep track of the percentage of time you spend on each page. Over time, the ratio settles and the scores give us some measure of the relative importance of these pages. You spend the most time on Ben, so Ben is ranked first, followed by Amy, then Dan, and lastly Chris.

It might seem like there's an easy way to beat the system, just make 100 pages all linking to your website. Now you get 100 full votes and you'll always rank on top, but that is not the case. While during their first few steps, they might make your page seem important, none of the other websites link to them. So over many steps, their contributions don't matter. You might have many links, but they're not quality links, so they don't affect the algorithm.

- But there is still one problem, though, not all pages are connected. In networks like this one, a random server can get stuck in a loop, never reaching the rest of the web. So to fix this, we can set a rule that 85% of the time, our random server just follows a link like normal. But then for about 15% of the time, they just jump to a page at random. This damping factor makes sure that we explore all possible parts of the web without ever getting stuck.

By using Markov chains, Page and Brin had built a better search engine, and they called it PageRank. - Because it's talking about how pages react, webpages react with each other and also 'cause the founder's name is Larry Page, so he snuck that in. - With PageRank, Google got much better search results, often getting you to the site you were looking for in one go. Although, to some, this sounded like a terrible idea.

- Others said, "Oh, well you're telling me you get a search that will get the right result on the first answer? Well, I don't want that because if it takes them three or four chances, searches to get the right answer, then I have three or four chances to show ads, and if you get 'em the answer right away, I'm just gonna lose them. So, you know, I don't see why better search is better." - But Page and Brin disagreed.

They were convinced that if their product was far superior, then people would flock to it. - I would say it actually is a democracy that works. If all pages were equal, anybody can manufacture as many pages as they want. I can set up a billion pages in my server tomorrow. We shouldn't treat them all as equal. Just looking at the data out of curiosity, we found that we had technology to do a better job of search, and we realized how impactful having great search can be.

- And so, in 1998, they launched their new search engine to take on Yahoo. Initially, they called it BackRub, after the backlinks it analyzed, but then they realized that maybe that's not the most attractive name. Now, their ambitions were big to essentially index all the pages on the internet, and they needed a name equally as big. So they thought of the largest number they could think of, 10 to the power of 100, a googol.

But then when trying to register their domain, they accidentally misspelled it. And so, Google was born. (dramatic music) Over the next four years, Google overthrew Yahoo to become the most used search engine. - Everyone who knows the internet almost certainly knows Google. - Googling is like oxygen to teenagers. - [Casper] And today, Alphabet, which is Google's parent company, is worth around $2 trillion. - When Google makes even the slightest change in its algorithms, it can have huge effects.

- Google. - Google. - Google. - Google. - They're on fire. And the reason why they're on fire is because they're focused and they're more focused than Yahoo who does search, they're more focused than Microsoft who does search with Bing. Yahoo has lots of traffic, they always have, they have some really great properties, but I don't think Yahoo is the go-to place, you know. - And at the heart of this trillion dollar algorithm is a Markov chain, which only looks at the current state to predict what's going to happen next. But in the 1940s, Claude Shannon, the father of information theory, started asking a different question.