In recent years, the penetration of artificial intelligence technology into different areas of the industry has increased rapidly. Therefore, the big data system is the foundation for realizing today’s data-driven AI, and therefore it becomes vital. This course is dedicated to guiding students to learn the basic concepts of big data systems, covering how to effectively store, process and analyze data. We start from the general principles of distributed system design; then we provide a framework on how to expand storage, computing and network functions in big data systems; finally, in order to make these design principles easy to follow, our case studies use real industrial systems To demonstrate how the basic design principles are applied to actual systems and how to analyze their performance and limitations.
How does the Internet work? The video you are watching now traveled thousands of miles from a Google data center to reach you. Let’s learn how the Internet works by getting to understand the details of this data’s incredible journey.
For many organization these days moving to the cloud is inevitable. However, it can be overwhelming for some businesses to completely migrate their traditional workloads to the cloud.
Check out this quick overview video with Andrea Crawford as she goes through how migrating to the cloud works and explains the many benefits that are derived from it.
Take a Google data center tour in 360° — brought to you by the Google Cloud Platform team.
Learn about the massive scale, the incredible attention to security and privacy, and the amazing efforts to make the data center extremely efficient and green.
How to view the 360° video:
Desktop using Google Chrome:
Use your mouse or trackpad to change your view while the video plays — highly recommend viewing in 4K
YouTube app on mobile:
Move your device around to look at all angles while the video plays
Google Cardboard:
Load the video in the YouTube app and tap on the cardboard icon when the video starts to play. Insert your phone in cardboard and enjoy. Cardboard is currently only supported by the YouTube app on Android. iOS support is coming soon!
Moore’s Law has been weakening since the early 2000s, as it continues to break down, data centers will struggle more and more to keep up with the increasing data demands of the 21st century. We believe a new category of microprocessor is needed, we’re calling it the Data Processing Unit, or DPU.
Today’s data centers are responsible for powering billions of connected devices for individuals, enterprises and institutions worldwide. Massive amounts of data are generated and consumed by these devices at unprecedented rates in today’s data-centric era. So, how will the next-generation data centers cope with this sky rocketing demand? To look forward, let us first look back at how data centers evolved. The original data center server architecture was not very different from a personal computer. At the heart of the server was the central processing unit, or CPU. Connected to the CPU were memory hard drives and a network interface controller, or NIC, which enables a connection to the network. Solid-state drives, or SSDs, were introduced when higher performance and more predictable access times were needed. In recent years, other elements such as graphics processing units, or GPUs, were added to the mix to run specialized computations tasks such as complex math functions far more quickly than a CPU ever could – this architecture is primarily compute-centric. In this architecture, the CPU has two roles to play: it plays its primary role of running applications, and at the same time, it plays the role of a data traffic controller – moving data between the network, the GPU, storage and others. This wasn’t too much of a problem in the past, when the network and storage were slow, and when CPUs could spend milliseconds on a single task. Further, the CPUs were doubling in performance every generation, thus never in the critical path. These days, SSDs are one hundred times faster than regular hard drives, and networks are thousands of times faster – but new generations of CPUs are no longer keeping pace. The traffic controller role is now highly intense. Not only was the CPU not designed for this role, in fact this role distracts the CPU from doing the work it does do well. To enable more efficient data centers, ones that can truly address the needs of the future, a new architecture beyond compute-centric is needed. One which is more aptly called, data-centric. The new architecture should liberate all server resources from being stranded behind the CPU, giving them direct access to the network – allowing them to focus on the tasks they do best. To enable the server resources to move data efficiently to and from each other, we are introducing the concept of a new type of processor known as the data processing unit, or DPU. First, the DPU should take on the role of a super-charged data traffic controller, offloading the CPU from this I/O intensive task, but doing it orders of magnitude more efficiently than the CPU. Specifically, the DPU should be adept at sending and receiving packets from the network, encrypting and compressing the immense amount of data moving around these servers, and running firewalls to protect servers against abuse. Second, the DPU should enable heterogenous compute and storage resources distributed across servers, to be pooled to maximize utilization, and in doing so, reduce the total cost of ownership (TCO) of the compute and storage resources. We believe data engines such as the DPU will enable data centers to reach the efficiencies and speeds necessary to empower the radical innovations that will soon change the world.
Data migration is the process of transferring data between one computer storage system to another and is crucial for businesses that need server or storage equipment replacements, maintenance or upgrades, application migration, website consolidation, disaster recovery, and data center relocation.
Check out this quick overview video with Katie Morgan as she goes through the three primary factors that you should be considering when you’re looking at a data transfer solution.
Joe Kava, VP of Google’s Data Center Operations, gives a tour inside a Google data center, and shares details about the security, sustainability and the core architecture of Google’s infrastructure.
