Netflix on AWS Cloud: Case Study
Netflix was originally a DVD shipping business where they would send out DVDs of your chosen programs to you. This was going well until 2008 where they experienced a major database loss and for 3 days could not ship out any DVDs to their customers. That was when the senior management at Netflix realized that they had to shift from continuous vertical scaling which
leads to single points of failure to a more reliable and scalable horizontal scaling system.
They chose Amazon Web Services despite having Amazon as a competitor (Amazon has their own streaming service known as Amazon Prime) because AWS provided them with the greatest scaling capabilities and the biggest set of available features. It took 7 years of migration for Netflix to shut down their last remaining data centers and move completely to the cloud.
Moving to the cloud has allowed Netflix to keep its existing members well engaged with overall viewing growing exponentially.
Application Monitoring on a Massive Scale :-
Netflix uses Amazon Web Services (AWS) for nearly all its computing and storage needs, including databases, analytics, recommendation engines, video transcoding, and more — hundreds of functions that in total use more than 100,000 server instances on AWS.
This results in an extremely complex and dynamic networking environment where applications are constantly communicating inside AWS and across the Internet.
Monitoring and optimizing its network is critical for Netflix to continue improving customer experience, increasing efficiency, and reducing costs. In particular, Netflix needed a solution for ingesting, augmenting, and analyzing the multiple terabytes of data its network generates daily in the form of virtual private cloud (VPC) flow logs. This would enable Netflix to identify performance-improvement opportunities, such as identifying apps that are communicating across regions and collocating them. The company would also be able to increase uptime by quickly detecting and mitigating application downtime.
John Bennett
Senior Software Engineer @ Netflix said,
Amazon Kinesis Data Streams processes multiple terabytes of log data each day, yet events show up in our analytics in seconds. We can discover and respond to issues in real-time, ensuring high availability, and great customer experience.
