Week #5 Functions behind the search engine
This week we had our blinders taken off and we were introduced to the topic of energy consumption with technology. I had never thought of something even remotely close to this conversation before. Now it has left me with so many questions, and thinking about the future.
For starters, who has thought about the energy consumption associated with a simple task of googling something? It is something so simple and reoccurring in everyday of our lives. We never stop to think about the actual process of it.
Step 1. Web Crawling:
Essentially, crawling is copying what is on web pages and repeatedly checking the multitude of pages to see if they are changed - and make a copy of any changes found.
The programs which have the job of doing this are variously referred to as robots, crawlers, and spiders.
Step 2. Indexing:
Once a spider has crawled a web page, the copy that is made is returned to the search engine and stored in a data center. These data centers are massive buildings, purposely built collections of servers which act as the living memory of the all the copies of webpages being made by the crawlers. For example, Google owns dozens of them spaced around the world, which it guards very closely and which are among the most hi-tech buildings in the world.
These buildings are referred to as the ‘Index’, and it is this data store which is organized and used to provide the search results you see on the search engine. Indexing is the process of organizing the masses of data and pages so they can be searched quickly for relevant results to your search query.
Step 3. The Algorithm:
Finally, there is a huge collection of web page copies which are being constantly updated and organized so we can quickly find what you are looking for. But we need a means by which they can be ranked in order of relevance to your search term – this is where the algorithm comes into play.
The algorithm is a very complex and lengthy equation which calculates a value for any given site in relation to a search term. We don’t know what the algorithm actually is, because search engines tend to keep this a closely guarded secret from competitors and from people looking to get to the top spots.
" Collectively, Google Search engines use about 2.26 million megawatt hours per year to power its global data center operations, which is equivalent to the power necessary to sustain 200,000 homes " - https://business.directenergy.com/blog/2017/november/powering-a-google-search
I hope you enjoyed my little rant, and you got your eyes opened up to the behind the scenes look at a google search! Until next time friends,
- DJ
Reference for Picture:
https://www.alternatememories.com/historical-events/movies/the-man-behind-the-curtain
For starters, who has thought about the energy consumption associated with a simple task of googling something? It is something so simple and reoccurring in everyday of our lives. We never stop to think about the actual process of it.
There are 3 functions that happen behind the curtain whenever someone presses the "Enter" tab on their smartphone, laptop, tablet, and or desktop. The complexity of the process will surprise you.
Essentially, crawling is copying what is on web pages and repeatedly checking the multitude of pages to see if they are changed - and make a copy of any changes found.
The programs which have the job of doing this are variously referred to as robots, crawlers, and spiders.
Step 2. Indexing:
Once a spider has crawled a web page, the copy that is made is returned to the search engine and stored in a data center. These data centers are massive buildings, purposely built collections of servers which act as the living memory of the all the copies of webpages being made by the crawlers. For example, Google owns dozens of them spaced around the world, which it guards very closely and which are among the most hi-tech buildings in the world.
These buildings are referred to as the ‘Index’, and it is this data store which is organized and used to provide the search results you see on the search engine. Indexing is the process of organizing the masses of data and pages so they can be searched quickly for relevant results to your search query.
Step 3. The Algorithm:
Finally, there is a huge collection of web page copies which are being constantly updated and organized so we can quickly find what you are looking for. But we need a means by which they can be ranked in order of relevance to your search term – this is where the algorithm comes into play.
The algorithm is a very complex and lengthy equation which calculates a value for any given site in relation to a search term. We don’t know what the algorithm actually is, because search engines tend to keep this a closely guarded secret from competitors and from people looking to get to the top spots.
That's all great, but whats the point Drew?
My point is, a simple google search really isn't as simple as you think it is. We are only seeing the results like that of the Wizard of Oz - we blindly and almost inconsiderately don't think about whats happening behind The Great Oz's curtain. These massive storage units of ever changing information in the "Indexing" stage use up an incredible amount of energy. And they need to be on 24/7, 365 days of the year. Otherwise, you will be running your search and getting nothing.
One Google search, uses an equivalent amount of energy as it does to turn on a 60 watt bulb for about 17 seconds. Think about how many times you individually use the google search engine in a year., Now think about how many other people in the world who also use it. That number gets scary big, scary fast. I found this quote while I used up some energy and Googled "Googles search engine power usage"." Collectively, Google Search engines use about 2.26 million megawatt hours per year to power its global data center operations, which is equivalent to the power necessary to sustain 200,000 homes " - https://business.directenergy.com/blog/2017/november/powering-a-google-search
I hope you enjoyed my little rant, and you got your eyes opened up to the behind the scenes look at a google search! Until next time friends,
- DJ
Reference for Picture:
https://www.alternatememories.com/historical-events/movies/the-man-behind-the-curtain
Great post its insane how much is done out there that we are unaware of. To us a simple google search doesn't cross much for a thought
ReplyDelete