Twitter plots Hadoop cluster migration to Google Cloud Platform

Twitter has outlined plans to maneuver a sizeable slice of its Hadoop data-processing platform to the Google Cloud to spice up the resiliency of the social media web site’s underlying infrastructure.

The firm is understood to function a number of Hadoop clusters, containing greater than 300 petabytes of knowledge, with the agency occurring file in January 2017 (through its engineering weblog) to substantiate that the most important of them contains 10,000 nodes.

These clusters have historically run in Twitter’s personal datacentres, and are used to hold out knowledge warehousing and processing duties, in addition to supporting the real-time facets of the micro-blogging platform’s interior workings.

In a follow-up submit, printed on three May 2018, the corporate stated it’s now working with Google to shift its chilly storage and versatile compute Hadoop clusters into the search large’s cloud platform.

“Over the past few years, we have been assessing our platform and infrastructure needs to make sure we are well positioned to keep up with the growing needs of our service,” wrote Twitter CTO Parag Agrawal, within the weblog submit.

“This migration, when full, will allow quicker capability provisioning; elevated flexibility; entry to a broader ecosystem of instruments and providers; enhancements to safety; and enhanced catastrophe restoration capabilities.

“Architecturally, we will also be able to separate compute and storage for this class of Hadoop workloads, which has a number of long-term scaling and operational benefits.”

Brian Stevens, CTO of Google Cloud, stated the partnership ought to pave the best way for even better ranges of technological collaboration between the 2 corporations in years to return.

“There is strong alignment with Twitter’s engineering strategy to meet the demands of its platform and the services Google Cloud offers at a global scale,” stated Stevens.

“Google Cloud Platform’s data solutions and trusted infrastructure will provide Twitter with the technical flexibility and consistency that its platform requires, and we look forward to an ongoing technical collaboration with their team.”

News of the tech tie-up between the 2 corporations coincides with Twitter’s admission bug in its methods could have resulted in some consumer passwords being saved insecurely. While the corporate claims there are not any indicators that the affected passwords have been misused or breached, Twitter is advising customers to contemplate altering their login particulars as a precaution.


Publish Date: 2018-05-04 07:20:00

Source hyperlink



    Leave a Comment


    Welcome! Login in to your account

    Remember me Lost your password?

    Lost Password