From the course: Advanced Node.js

Why streams?

“

- [Instructor] Streams are very important to Node.js and as an Node.js developer, you're going to want to know your way around the stream interface. To exemplify how streams work in Node.js, we're going to use an analogy. Consider a very large bucket of water. So you have a giant bucket of water and let's say you have to move this giant bucket of water into another giant bucket. So, we'll refer to our bucket of water as the source and the bucket that we need to move the water to as the destination. Now, if you wanted to build a business around moving water from the source to the destination, you're going to need some space. So there we go. Your business has a factory and in that factory, you have an empty, giant bucket with a B on it. That stands for buffer. So when it's times for you to actually move the water from the source to the destination, you can pull the water into your factory, pour all of it into your buffer bucket, and pull the destination bucket into the factory, and pour all of the water in your buffer bucket into the destination. There's a solution. But there's got be a better way, and there is. A hose. What if we were able to hook a hose up to the source bucket and pipe all of the water through the hose to the destination bucket? We will have created a stream, and water will move bit by bit from the source to the destination bucket. Now, in Node.js, we're not in the business of moving water, we're in the business of moving data, but we have a similar problem to solve. Let's take a look at a couple examples. So I'm inside of chapter two, chapter two, lesson one. Within the start folder, you'll notice there's a couple of files. We have stream.js file, a buffer.js file, and a little mp4 movie of a powder-day here in Lake Tahoe. So let's start off by taking a look at the buffer.js file. Now again, what we're trying to do is move data from a source to a destination. The source is the powder-day.mp4 video, and we've declared a reference to that on line three, and the destination is going to be a HTTP response. The actual data is getting sent to the destination on line 12 when we end the response. So, we're creating a HTTP server that runs on Port 3000 and it's how we get the file that matters. So we're using the FS.read file, which is going to read the entire contents of the MP4 video into a single variable. That variable is data, so that's our buffer, and we're sending the entire data back on line 12, once we have the entire contents of the movie loaded. So let's take a look at what this does to memory. We're gonna come over to my desktop, and I have my activity monitor open. And you can find that simply by searching for activity monitor. And what we're gonna be checking is the memory. So I'm gonna go to the memory tab, and I'm gonna do a search for any node processes. So we don't have any running now, but we're about to start one. I'm gonna come over to my terminal, and I'm gonna go ahead and start the node process for buffer.js by typing node - -trace _gc buffer .js. The trace gc command is gonna go ahead and trace garbage collection on this process, so we can see how our garbage collector is working. So great, now that we have this running on localhost:3000, you can actually see in the activity monitor, we have a node process. We're using memory, awesome. So let's go ahead and go to a web browser, and make some requests for our server. So here, it's running on localhost:3000. I have five browser tabs open, and I'm gonna make five requests. I'm turning the sound off in each request. There we go, so. Request four, and then let's make one more. Request five. There we go. So we have five requests running, all getting the video. So let's take a look at what's happening. So inside of the terminal window, you're gonna notice two types of garbage collection. We have Scavenge, and we have Mark-sweep. And you're gonna notice that we have actually, a lot of these Mark-sweeps. So this Mark-sweep is actually a much bigger deal because Mark-sweep is gonna stop your node process to clean up a lot of garbage. The Scavenge doesn't clean up as much garbage. We're also gonna notice that we're using 157 megabytes of memory. So, that's how we handle this with the buffer. I'm gonna go ahead and give this a stop, by hitting control+C. And we'll go ahead and clear the screen. We'll also notice our node process disappear in the activity monitor. So let's go ahead and take a look at the stream example. So, the only difference in the stream example is on line eight, we're creating a read stream with the file system. So that will read this file bit by bit, and send those bits to the client as soon as they get them. So let's go ahead and try this again. Let's come back out to our terminal. And this time, we're gonna node - -trace _gc stream.js. And we have that running in localhost:3000. So I'll come over here to localhost, and we're gonna make the same five requests. One, two, three, four, and five. So what you'll notice is we have some Scavenge garbage collection, but we don't have any Mark-sweeps. And again, Scavenge is a quicker version of the garbage collector that runs on a smaller bit of memory. You can almost think of it as like your short-term memory for your application. The Mark-sweep is a bigger deal, and we don't see any of those. Additionally, if we come over to the activity monitor, you will notice that we're only using 40 megabytes of memory. So that's significantly less memory. And the reason is we aren't loading the entire contents of each video file before we're sending it, we're sending it bit by bit. So, in order to create performant applications, streams are absolutely essential.

Contents