Google combines big data and search algorithms to crawl blockchains

Ryan is a senior editor at TechForge Media with over a decade of experience covering the latest technology and interviewing leading industry figures. He can often be sighted at tech conferences with a strong coffee in one hand and a laptop in the other. If it's geeky, he’s probably into it. Find him on Twitter (@Gadget_Ry) or Mastodon (

Google, to a certain extent, has been behind other tech organisations in terms of managing and creating blockchains. This is because till date, the search engine giant was more about big data and search algorithms. But now it has combined the two to make blockchains searchable via its new suite of software tools to search the data.

For Allen Day — who works as a senior developer advocate for Google Cloud – customers demand before products even exist, and he believes that the accessible blockchain would be the new upcoming trend. And that is why he started to feed the entire Bitcoin and Ethereum blockchains into BigQuery, which is Google’s fully managed enterprise data warehouse for large-scale data analytics. Then came the time to create a suite of software tools to search the data; this ongoing project, known as Blockchain ETL (extract, transform, load), became quite a popular name in the developer communities.

Blockchain ETL was merely a word of mouth hit amongst crypto-coders and has given rise to over 500 projects since the suite was created. These projects range from bitcoin price prediction to wealth disparity amongst Ether wallets. Day is continuously focusing on who is using the blockchain and in what manner it is being used. He is keeping tabs of the use of AI on the Ethereum blockchain just to make sure whether crypto-exchange bots are not wash-trading.

Day says: “In the future, moving more economic activity on chain won’t just require a consensus level of trust. It will require having some trust in knowing about who it is you’re actually interacting with.”

Earlier, searching the blockchain was understood as using a block-explorer in order to retrieve data on a known individual transaction number. Now Blockchain ETL allows generalised searches of entire blockchains. Apart from Bitcoin and Ethereum, Day’s team is now adding other cryptocurrencies like Litecoin, Zcash, Dash, Bitcoin Cash, Ethereum Classic, and Dogecoin to BigQuery.

Day has also used his tools to analyse the hard-fork of 2017, which gave birth to the Bitcoin Cash blockchain. He found that instead of increasing micro-transactions as hoped, the majority of the currency was being accumulated by big holders.

Day explains: “I’m very interested to quantify what’s happening so that we can see where the legitimate use cases are for blockchain. Then we can move to the next use case and develop out what these technologies are really appropriate for.”

Interested in hearing leading global brands discuss subjects like this in person? Find out more at the Blockchain Expo World Series, Global, Europe and North America.

Tags: , , ,

View Comments
Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *