Best pixel streaming solution for 3D applications

December 7, 2022
6
 min read

Benchmarking different pixel streaming solutions for 3D applications

This article helps to explain in more detail the differences in performance between the 3 main pixel streaming solutions that are actively in use with Flaneer:

NICE DCV (by AWS)

This was the original streaming protocol we used and still use at Flaneer. It can be used both in the browser and with a local client. These tests were performed with the local client.

Reemo

This is the new streaming protocol that we are using on Flaneer as the default, you can find the reasons why in our blog post. It is a browser only protocol, so these tests were performed in the browser.

Parsec

This is a premium streaming protocol that some of our users elect to use since they find it has the best performance. It is a local client only protocol, so these tests were done in the local client.

This article has some limitations that we want to be transparent about upfront. First and foremost, that the data comes from the apps themselves. We are happy to work with the assumption that these apps are accurately reporting this information, but we wanted to be transparent about the source of information.

Secondly, there are so many moving variables when it comes to measuring streaming performance that it is hard to get really robust numbers. We used the tool Netlimiter 4 to limit the connection and viewed performance while watching the same one-minute video clip.

Thirdly, as you will see in the data, there is not a simple way to do quality of the stream vs internet speed. There are many different metrics that may concern you more due to the conditions of your home network or the machine that you are using in the cloud.

Some basics about pixel streaming

Frames per Second (FPS)

This is the amount of displayed frames in a given second. This gives a good idea of the “smoothness” of the streaming experience.  For most use cases 30 fps is going to be perfect, for a responsive and smooth streaming experience 60fps is preferred.

Latency (ms)

Latency is simply the time between capturing a frame and displaying it. It will affect how responsive the streaming experience is, the lower the latency, the faster the response of the streaming PC will be. For example, if I click a button, how long until I see the machine respond to the click. The values you will see below are measured in milliseconds.

Bandwidth (Mb/s)

Bandwidth is the amount of data that the streaming app is consuming on a given time period. For the statistics in the article, this is in megabits per second. Bandwidth will be of particular interest to those with limited connections.

If you want to know more about streaming technology, we explain it all in this article

How did I benchmark the 3 pixel streaming solutions?

The methodology is quite simple, I booted up the same Iron tier machine on Flaneer for all three pixel streaming solutions. The Iron tier machine is a great all round performer for GPU users on Flaneer. It is not overly powerful, but it is clearly capable of running game engines, 3D art work and video editing.

Then I set the resolution of the streaming to be 1080p and the framerate to 60 FPS target. A note here is that the target is interpreted in different ways by the different software. The main thing to note is that Perforce will aggressively seek to make that target, so the quality will drop until 60fps is reached.

Once each machine was streaming, I played the same 60fps 1080p video clip for sixty seconds to get an average of the performance over that time. I repeated this test while limiting my internet speed to 100Mb, 70Mb, 30Mb and 10Mb. These values were chosen as they represent a “fast” connection, the global median broadband, the global median mobile and our recommended minimum. (Source: https://www.speedtest.net/global-index)

For quality assessment, things can be quite difficult. Internally, when testing our own streaming protocol (more on that in the changelogs) we can use something called the PSNR to measure quality empirically, however to do this we need to locally record the screen, then record the streamed image and compare the two. This was outside the scope of this comparison, so we have a video where you can see the quality for yourself.

Video Comparison of Pixel Streaming

How those pixel streaming solutions performed?

So with all that preamble out the way, the bit that everyone will skip to is here! Here are the raw data tables:

NICE DCV (by AWS)

Reemo

Parsec

In conclusion, what’s the best pixel streaming solution?

OK, so, there are 3 clear personalities we can see shining through here. DCV seems to be aiming to not majorly rock the boat on any of the metrics, it will try to keep FPS at 60 with a low enough latency while using a fairly conservative amount of bandwidth. This makes it a great choice for an all round balanced streaming experience. In comparison, Reemo is clearly aiming for a low latency low bandwidth approach and then sacrificing framerate to consistently achieve this. Whereas, Parsec is happy to eat as much bandwidth as it needs to get 60fps consistently.

While it clearly is lagging behind in terms of FPS, Reemo is miles ahead in terms of both latency and bandwidth. On metered connections, or connections with low speeds it is a clear choice, as well as being the most responsive to user inputs. There is a key metric of 16.7 ms, which is the frame time for 60fps. So in some cases DCV and Parsec would be displaying frames at 60fps, but the inputs would be slower than 60fps, so it would look smoother, but it would not feel more responsive.

Of course, if you are sitting at home on a high speed internet connection without a care in the world about bandwidth consumption, perhaps one of the heavier duty pixel streaming may indeed be preferable for you.

In conclusion…again, what’s the best pixel streaming solution for you?

OK, so there is no clear winner here, and we talked a little bit there about the different network conditions that may cause you to lean from one of the solutions to another. However, use case is also going to be a big influence on how you might want to make that decision as well. I'm going to fire off a quick list of the best solution for you in different use cases, assuming you have a 70Mb connection or better.

  • Watching videos - During the video comparison, you can see that DCV and Parsec look the same while watching video. Since DCV can look like that and use less bandwidth, it is the one to go for here.
  • Gaming - Parsec seems to have the most support in that particular use case. It can do 60fps at <17ms latency, so it will look and feel right, and the UX of Parsec is geared towards gaming as a primary use case.
  • Office Work - There are no real issues with any of the solutions compared here. So why not pick Reemo and use less bandwidth!
  • 3D Work - For 3D work, (Unreal Engine, Unity, Maya, Blender, CAD, etc.) DCV is a great choice. The image quality and consistency is there, and the easy file upload/download in the local client is really useful on these types of tasks.
  • Video Editing - Video editing is also great on DCV for the same reasons. In fact, at Flaneer we edit our videos on our own machines using DCV! However, there is a non-free tier of Parsec that allows for more accurate colour streaming that may be of interest to those doing high fidelity colour work.

Best computers to run Unreal engine 5

Want to launch an Nvidia A10G 24GB RAM in a minute?
Silver
16GB GPU Tesla T4
8 cores
32GB RAM
€158 /month
Try this machine
Sylvia
24GB GPU NVIDIA A10G
8 cores
32GB RAM
€269 /month
Try this machine
Davida
24GB GPU NVIDIA A10G
16 cores
64GB RAM
€333 /month
Try this machine

Latest articles

Browse all