Best pixel streaming solution for 3D applications
January 30, 2023
6
min read
This article helps to explain in more detail the differences in performance between the 3 main pixel streaming solutions that are actively in use with Flaneer:
This was the original streaming protocol we used and still use at Flaneer. It can be used both in the browser and with a local client. These tests were performed with the local client.
This is the new streaming protocol that we are using on Flaneer as the default, you can find the reasons why in our blog post. It is a browser only protocol, so these tests were performed in the browser.
This is a premium streaming protocol that some of our users elect to use since they find it has the best performance. It is a local client only protocol, so these tests were done in the local client.
This article has some limitations that we want to be transparent about upfront. First and foremost, that the data comes from the apps themselves. We are happy to work with the assumption that these apps are accurately reporting this information, but we wanted to be transparent about the source of information.
Secondly, there are so many moving variables when it comes to measuring streaming performance that it is hard to get really robust numbers. We used the tool Netlimiter 4 to limit the connection and viewed performance while watching the same one-minute video clip.
Thirdly, as you will see in the data, there is not a simple way to do quality of the stream vs internet speed. There are many different metrics that may concern you more due to the conditions of your home network or the machine that you are using in the cloud.
This is the amount of displayed frames in a given second. This gives a good idea of the “smoothness” of the streaming experience. For most use cases 30 fps is going to be perfect, for a responsive and smooth streaming experience 60fps is preferred.
Latency is simply the time between capturing a frame and displaying it. It will affect how responsive the streaming experience is, the lower the latency, the faster the response of the streaming PC will be. For example, if I click a button, how long until I see the machine respond to the click. The values you will see below are measured in milliseconds.
Bandwidth is the amount of data that the streaming app is consuming on a given time period. For the statistics in the article, this is in megabits per second. Bandwidth will be of particular interest to those with limited connections.
If you want to know more about streaming technology, we explain it all in this article
The methodology is quite simple, I booted up the same Iron tier machine on Flaneer for all three pixel streaming solutions. The Iron tier machine is a great all round performer for GPU users on Flaneer. It is not overly powerful, but it is clearly capable of running game engines, 3D art work and video editing.
Then I set the resolution of the streaming to be 1080p and the framerate to 60 FPS target. A note here is that the target is interpreted in different ways by the different software. The main thing to note is that Perforce will aggressively seek to make that target, so the quality will drop until 60fps is reached.
Once each machine was streaming, I played the same 60fps 1080p video clip for sixty seconds to get an average of the performance over that time. I repeated this test while limiting my internet speed to 100Mb, 70Mb, 30Mb and 10Mb. These values were chosen as they represent a “fast” connection, the global median broadband, the global median mobile and our recommended minimum. (Source: https://www.speedtest.net/global-index)
For quality assessment, things can be quite difficult. Internally, when testing our own streaming protocol (more on that in the changelogs) we can use something called the PSNR to measure quality empirically, however to do this we need to locally record the screen, then record the streamed image and compare the two. This was outside the scope of this comparison, so we have a video where you can see the quality for yourself.
So with all that preamble out the way, the bit that everyone will skip to is here! Here are the raw data tables:
OK, so, there are 3 clear personalities we can see shining through here. DCV seems to be aiming to not majorly rock the boat on any of the metrics, it will try to keep FPS at 60 with a low enough latency while using a fairly conservative amount of bandwidth. This makes it a great choice for an all round balanced streaming experience. In comparison, Reemo is clearly aiming for a low latency low bandwidth approach and then sacrificing framerate to consistently achieve this. Whereas, Parsec is happy to eat as much bandwidth as it needs to get 60fps consistently.
While it clearly is lagging behind in terms of FPS, Reemo is miles ahead in terms of both latency and bandwidth. On metered connections, or connections with low speeds it is a clear choice, as well as being the most responsive to user inputs. There is a key metric of 16.7 ms, which is the frame time for 60fps. So in some cases DCV and Parsec would be displaying frames at 60fps, but the inputs would be slower than 60fps, so it would look smoother, but it would not feel more responsive.
Of course, if you are sitting at home on a high speed internet connection without a care in the world about bandwidth consumption, perhaps one of the heavier duty pixel streaming may indeed be preferable for you.
OK, so there is no clear winner here, and we talked a little bit there about the different network conditions that may cause you to lean from one of the solutions to another. However, use case is also going to be a big influence on how you might want to make that decision as well. I'm going to fire off a quick list of the best solution for you in different use cases, assuming you have a 70Mb connection or better.
