{"id":604,"date":"2015-03-23T11:25:10","date_gmt":"2015-03-23T11:25:10","guid":{"rendered":"http:\/\/adriangrigoras.com\/blog\/?p=604"},"modified":"2015-03-23T11:25:10","modified_gmt":"2015-03-23T11:25:10","slug":"scaling-mysql-cloud-vitess-kubernetes","status":"publish","type":"post","link":"https:\/\/adriangrigoras.com\/blog\/scaling-mysql-cloud-vitess-kubernetes\/","title":{"rendered":"Scaling MySQL in the cloud with Vitess and Kubernetes"},"content":{"rendered":"<p>Your new website is growing exponentially. After a few rounds of high fives, you start scaling to meet this unexpected demand. While you can always add more front-end servers, eventually your database becomes a bottleneck, which leads you to . . .<\/p>\n<ul>\n<li>Add more replicas for better read throughput and data durability<\/li>\n<li>Introduce sharding to scale your write throughput and let your data set grow beyond a single machine<\/li>\n<li>Create separate replica pools for batch jobs and backups, to isolate them from live traffic<\/li>\n<li>Clone the whole deployment into multiple datacenters worldwide for disaster recovery and lower latency<\/li>\n<\/ul>\n<p>At YouTube, we went on that <a href=\"https:\/\/www.youtube.com\/watch?v=5yDO-tmIoXY&amp;feature=youtu.be\" target=\"_blank\">journey<\/a> as we scaled our MySQL deployment, which today handles the metadata for billions of daily video views and <a href=\"http:\/\/www.youtube.com\/yt\/press\/statistics.html\" target=\"_blank\">300 hours of new video uploads per minute<\/a>. To do this, we developed the <a href=\"http:\/\/vitess.io\/\" target=\"_blank\">Vitess<\/a>platform, which addresses scaling challenges while hiding the associated complexity from the application layer.<\/p>\n<p>Vitess is available as an <a href=\"https:\/\/github.com\/youtube\/vitess\" target=\"_blank\">open-source project<\/a> and runs best in a containerized environment. With <a href=\"http:\/\/kubernetes.io\/\" target=\"_blank\">Kubernetes<\/a> and<a href=\"https:\/\/cloud.google.com\/container-engine\/\" target=\"_blank\">Google Container Engine<\/a> as your <a href=\"http:\/\/googlecloudplatform.blogspot.com\/2015\/01\/what-makes-a-container-cluster.html\" target=\"_blank\">container cluster manager<\/a>, it&#8217;s now a lot easier to get started. We\u2019ve created a single deployment configuration for Vitess that works on <a href=\"http:\/\/kubernetes.io\/gettingstarted\/\" target=\"_blank\">any platform that Kubernetes supports<\/a>.<\/p>\n<p>In addition to being easy to deploy in a container cluster, Vitess also takes full advantage of the benefits offered by a container cluster manager, in particular:<\/p>\n<ul>\n<li><b>Horizontal scaling<\/b> \u2013 add capacity by launching additional nodes rather than making one huge node<\/li>\n<li><b>Dynamic placement<\/b> \u2013 let the cluster manager schedule Vitess containers wherever it wants<\/li>\n<li><b>Declarative specification<\/b> \u2013 describe your desired end state, and let the cluster manager create it<\/li>\n<li><b>Self-healing components<\/b> \u2013 recover automatically from machine failures<\/li>\n<\/ul>\n<p>In this environment, Vitess provides a MySQL storage layer with improved durability, scalability, and manageability.<\/p>\n<p>We&#8217;re just getting started with this integration, but you can already <a href=\"http:\/\/vitess.io\/getting-started\/\" target=\"_blank\">run Vitess on Kubernetes<\/a> yourself. For more on Vitess, check out our <a href=\"http:\/\/vitess.io\/\" target=\"_blank\">website<\/a>, ask questions on our <a href=\"https:\/\/groups.google.com\/forum\/#!forum\/vitess\" target=\"_blank\">forum<\/a>, or join us on <a href=\"https:\/\/github.com\/youtube\/vitess\" target=\"_blank\">GitHub<\/a>. In particular, take a look at our overview to understand the trade-offs of Vitess versus NoSQL solutions and fully-managed MySQL solutions like <a href=\"https:\/\/cloud.google.com\/sql\/\" target=\"_blank\">Google Cloud SQL<\/a>.<\/p>\n<p>-Posted by Anthony Yeh, Software Engineer, YouTube<\/p>\n<p>source:\u00a0googlecloudplatform.blogspot.com<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Your new website is growing exponentially. After a few rounds of high fives, you start scaling to meet this unexpected demand. While you can always add more front-end servers, eventually your database becomes a bottleneck, which leads you to . . . Add more replicas for better read throughput and data durability Introduce sharding to\u2026 <span class=\"read-more\"><a href=\"https:\/\/adriangrigoras.com\/blog\/scaling-mysql-cloud-vitess-kubernetes\/\">Read More &raquo;<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30],"tags":[],"class_list":["post-604","post","type-post","status-publish","format-standard","hentry","category-architecture"],"_links":{"self":[{"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/posts\/604","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/comments?post=604"}],"version-history":[{"count":1,"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/posts\/604\/revisions"}],"predecessor-version":[{"id":605,"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/posts\/604\/revisions\/605"}],"wp:attachment":[{"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/media?parent=604"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/categories?post=604"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/adriangrigoras.com\/blog\/wp-json\/wp\/v2\/tags?post=604"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}