How much does your arcade stick lag? Arcade stick input lag testing & results thread

12021222325

Comments

  • par124par124 Joined: Posts: 12
    First of all that setup's really cool noodalls! Looking forward to seeing more of your results.

    I really have no idea how the variation works (whether it's from the console, the game or the controller, etc), but I suggested in my post earlier on this page that I think it might be helpful to include something like a standard deviation value for the sets of delay measurements rather than just a straight average latency value when evaluating PCBs, because even the evaluation method from the OP (determining an "average" from "Yes"s and "No"s) occasionally suggested significant latency variance in how some controllers' inputs are read.

    I think this certainly makes comparing controllers more complicated. Like if controller A has an average latency of 8ms and a very small standard deviation, and controller B has an average latency of 5ms but a huge standard deviation, it would be difficult for me to say which one I'd call "preferable" in terms of lag. I think I'd prefer the one that looks cooler or feels comfier.
  • noodallsnoodalls Joined: Posts: 404
    I have a feeling that some games vary with regards to timing a lot more than others. Blazblue seems very consistent and that's why I'm using that predominantly.

    I have a new idea of how to generate larger volumes of meaningful data quickly and easily, and will try to do a trial run soon.
  • noodallsnoodalls Joined: Posts: 404


    This shows how inputs will be accepted all across the frame using a TE2, but only at certain times with the UFB.

    Note, I do need to go back and make the data a bit clearer. Some of the results at the end of the frame for the TE2 (16 and 2/3) are actually from a frame earlier, so should read as 17 and 18 to make things clearer.
  • noodallsnoodalls Joined: Posts: 404


    So pretty much this. If I had to guess, and I'd be happy to be proven wrong, perhaps the TE2 is polling less than once per frame, but can take less time than the UFB to process the information, whereas the UFB is polling more than once per frame (hence the more frequent results closer to 2msec, and fewer further out) but does take a touch longer to process (based on the fastest result).
  • noodallsnoodalls Joined: Posts: 404
    edited October 2016
  • noodallsnoodalls Joined: Posts: 404


    Got around to testing PS4. Had been waiting until I got a PS360+ Ver 2.10.

    The UFB did come out on top, but what was surprising to me is the number of results. The other sticks gave about the expected number of results (~6000 frames of data, every 6th frame an input occurs, but only lasts for 1msec, i.e. 1/16.66 of a frame. Therefore for ~6000 frames you would expect ~60 successful inputs) in the 60s and 70s. The UFB had 240 results.
  • noodallsnoodalls Joined: Posts: 404
    Numbers refer to how many miliseconds back from the bottom of the screen two frames before the input and animation appears occur.

  • noodallsnoodalls Joined: Posts: 404
    edited October 2016
    par124 wrote: »
    First of all that setup's really cool noodalls! Looking forward to seeing more of your results.

    I really have no idea how the variation works (whether it's from the console, the game or the controller, etc), but I suggested in my post earlier on this page that I think it might be helpful to include something like a standard deviation value for the sets of delay measurements rather than just a straight average latency value when evaluating PCBs, because even the evaluation method from the OP (determining an "average" from "Yes"s and "No"s) occasionally suggested significant latency variance in how some controllers' inputs are read.

    I think this certainly makes comparing controllers more complicated. Like if controller A has an average latency of 8ms and a very small standard deviation, and controller B has an average latency of 5ms but a huge standard deviation, it would be difficult for me to say which one I'd call "preferable" in terms of lag. I think I'd prefer the one that looks cooler or feels comfier.

    As always, happy to hear other people's thoughts.

    Looking at the Hori vs the Akishop boards. What I think may be obtainable from this data is 1. how quickly the controller polls for changed inputs and 2. how quickly it can convey that data to the PS4.


    Let's pretend we had the perfect stick. If it took the absolute least amount of time to poll (or conversely polled at the highest rate), and transferred that information to the PS4 in the least amount of time, it would have 100% of values in the 1msec range (assuming that is the fastest possible, thus far it has been the fastest observed).

    Now, none of the sticks here meet that goal. One time only the Aki achieved 1msec results. On average it is 2msec. However there is a spread of results. This suggests to me that the polling rate is less than once per millisecond. This applies to all sticks. And if you go back to my Xbox TE2 the results were spread across almost the entire frame.

    The other thing to comment on, the way this test works is every sixth frame I turn the button on for 1msec. Now, for the HRAP and Aki tests this resulted in about the right number of results. e.g. for the Aki 2.10 the test lasted for 6367 frames (time for the input to scroll all the way to the starting point). If we divide this by 16.66 and again by 6 the results is, 64, so we would expect 64 inputs over this period of time. We actually got 71, which is within just over 10% of the expected result. For the PS360+ old version we expected 133, got 139. For the HRAP V4 we expected 61 got 63. For the HRAPV we expected 61 got 75, so still somewhat close.

    For the UFB we expected 62 results. We got 240! So almost four times as many as expected. This to me suggests there is something fundamentally different about the way that the UFB is reading inputs to the other controllers.

    Post edited by noodalls on
  • noodallsnoodalls Joined: Posts: 404


    Edit - previous tweet had typo, so reposted.
  • mistahsnartmistahsnart Joined: Posts: 511
    *sets old Dual Shock 4 controllers on fire* Time to go shopping.
  • noodallsnoodalls Joined: Posts: 404
    *sets old Dual Shock 4 controllers on fire* Time to go shopping.

    My interpretation was the USB mode is slower.
  • MoonchildeMoonchilde Resident tech talk double poster Joined: Posts: 4,836
    I thought DS4 doesn't send data over USB, and only uses it for charging?
  • noodallsnoodalls Joined: Posts: 404
    Moonchilde wrote: »
    I thought DS4 doesn't send data over USB, and only uses it for charging?

  • MoonchildeMoonchilde Resident tech talk double poster Joined: Posts: 4,836
    Ah, so they changed it? Good on them, that was a silly restriction anyway.
  • noodallsnoodalls Joined: Posts: 404
    Only works with the new version of the controllers.
  • DarksakulDarksakul Your lack of faith disturbs me Joined: Posts: 22,046
    noodalls wrote: »
    Only works with the new version of the controllers.

    Only the new version, controllers? Does the console matter? Ether way I need to keep that in mind.
    "You must defeat Sheng Long to stand a chance."
  • noodallsnoodalls Joined: Posts: 404
    The option is there on the standard PS4 console. Settings --> devices --> controllers --> communication method

    The text underneath says "This setting is for the DUALSHOCK 4 (CUH-ZCT2 series only)."
  • MoonchildeMoonchilde Resident tech talk double poster Joined: Posts: 4,836
    :( Seriously?
  • AriesnoAriesno Seimitsu's warrior Joined: Posts: 402
    Somebody know about Qanba's Dragon input lag ?

    I will buy an Obsidian , i suppose it share the same Pcb or close to Dragon...


  • Seth HD RemixSeth HD Remix Hi my name is Seth, I play Ken Joined: Posts: 99
    edited January 3
    ^ ditto any data on those new quanbas input lag ? In the market and between this or medketz
    SFV id: kkmasters
    Psn id: SethTKP
    xbl : Seth HD Remix
    twitter: @SethTKP
    Insta: kk_masters
    twitch: kkmasters
  • noodallsnoodalls Joined: Posts: 404
    Would love to test them, but not going to buy one just to test.
  • noodallsnoodalls Joined: Posts: 404


    Will take a bit of explaining as to exactly how these numbers were established, but will edit that in later.
  • noodallsnoodalls Joined: Posts: 404
    So, the setup is the same as per previous times. Coloured band appears on the screen and a button on the stick/pad is triggered at the same time. I have used a 1ms input window to try to more accurately determine where the inputs are actually polled. When the latest input occurs, I capture a screen shot of this image, then count down to where the coloured bar is to determine how late in the frame the input has occurred. Converting this into ms is possible, and then 2F (33.33ms) is added for the next two frames until an animation change occurs.

    An HDMI to Component converter is used, in a separate test I've established that this adds in the order of .25ms of lag.

    Game is BBCPE as usually, chosen because it is rapid to respond, consistent and available across all the relevant platforms.

  • noodallsnoodalls Joined: Posts: 404


    This looked at how early in the frame an input could occur. This is useful as combined with the other data it gives some idea of either how wide the polling window is, or how frequently the sticks are polled. The other piece of data that would be useful is how many results were generated from each set, however I'll leave that for now.

    As you can see, at least 10 results were generated for each stick (5 earliest and 5 latest). I've captured that many again, but in the interest of time haven't analysed all of them.
  • saikofartsaikofart ex anorak Joined: Posts: 121
    edited January 4
    Starting to love your work @noodalls !

    So the wired USB mode for the dualshock 4 is slower than Bluetooth mode? Were your tests on ps4? I feel lied to by Sony!
  • noodallsnoodalls Joined: Posts: 404
    All tests are on PS4.
  • noodallsnoodalls Joined: Posts: 404
    Tried a new method of sending outputs to the sticks/pads/pcbs, to get a little more information out of each trial. I quite like the way that it works, so will write it up here.

    As always, the setup is an arduino sending signals to push/release buttons while simultaneously turning off parts of a component video signal, so that a watermark appears on the screen to demonstrate when the button was pushed. For this test I used the UFB, as it has been reliably the fastest responding device. I also used BBCPE, which is my standard game as it is fast, and available across multiple formats.

    What is different this time is that instead of a single input, I did two different inputs of different length. The cycle is 24 frames long, as this allows the BBCPE Ragna ducking animation to finish with no concerns of overlapping animation.

    As usual, the input advances by 25-50 microseconds on each subsequent cycle.

    The video demonstrating this is here.


    Now, it starts roughly at the top, ignore the first few cycles.






  • noodallsnoodalls Joined: Posts: 404
    Putting things together a little bit, most sticks/pcbs/pads I've looked at must have a fairly high polling rate, as the results tend to bunched together fairly closely.

    However, the MCCthulhu (on 10ms) and the TE2 don't seem to behave like this. Even between them, my impression is that the X1TE2 polls maybe every 10ms (not the exact number) whereas the MCCthulhu keeps the polling window open for 10ms. The difference being the TE2 can still achieves results about as fast as the UFB at times, whereas the MC seemed to never get faster than about 10ms slower than the 1ms polling time result.
  • noodallsnoodalls Joined: Posts: 404
    Explanation of input lag testing


    Demonstration of input lag testing (wireless vs wired DS4 controller)


    Video captured during the above video



    1P red left new DS4 wired in USB mode , responding with green bar near the top of the screen. Top = earlier in the frame = slower.



    2P blue right old DS4 usb cable connected for charging in wireless mode. Green bar later = faster.
  • hyperheavywinghyperheavywing Joined: Posts: 25
    Hello, I am wondering what is the best pcb board for pc? I mostly play SF5, It doesn't matter it is via direct input or x-input as long as it most responsive. Also curious if there is a method to reduce input lag on pc (i.e premium sata USB board?)
  • AriesnoAriesno Seimitsu's warrior Joined: Posts: 402
    Hori hrap VX , X360 seem one of the best.
  • hyperheavywinghyperheavywing Joined: Posts: 25
    Ariesno wrote: »
    Hori hrap VX , X360 seem one of the best.
    so pretty much 360 lag result is applied to PC ?
  • AriesnoAriesno Seimitsu's warrior Joined: Posts: 402
    I don't know about other 360 arcade stick but i sure Hori hrap Vx is my fastest arcade stick on both 360 and PC.
    It seem is faster than last ps4 Hrap 2017 too.

    https://youtu.be/ZYfndhvN7Qk
  • hyperheavywinghyperheavywing Joined: Posts: 25
    Ariesno wrote: »
    I don't know about other 360 arcade stick but i sure Hori hrap Vx is my fastest arcade stick on both 360 and PC.
    It seem is faster than last ps4 Hrap 2017 too.

    https://youtu.be/ZYfndhvN7Qk
    thank you for the info.
  • hyperheavywinghyperheavywing Joined: Posts: 25
    I am also curious if there is any test between Vx SA and xboxone sticks on PC (i.e Te2 Razer Atrox XB1) since XB1 support usb3
  • noodallsnoodalls Joined: Posts: 404
    Nikogel360 has been very busy on youtube producing comparisons between different sticks for a few years. I thought I'd try to compile that work in one post.
    https://twitter.com/nikogel360
    https://www.youtube.com/user/nikogel360/


    Testing with Ryu vs Ryu (square buttons wired together) on PS3 Ultra SF4.
    RAPV4 (PS3 mode) vs PSU2TE2 (PS3 mode) 3:2
    RAPV4 (PS4 mode) vs PSU2TE2 (PS3 mode) 10:0
    RAPV4 (PS4 mode) vs PSU2TE2 (PS4 mode) 1:3
    RAPV4 (PS3 mode) vs PSU2TE2 (PS4 mode) 0:20
    RAPV4 (PS4 mode) vs RAPV3-SA 4:6


    Testing with PC USF4
    RAPVXSA vs RAPV silent 隼 (PS3 mode) 93:0

    (more later)

    https://www.openrec.tv/movie/jPgYoNkL4AL
    PS4 SFV
    Testing with Chun Li vs Chun Li
    Brook UFB vs XCM Cross fire converter (ver 3.0) with RAPVX 12:0
    Brook UFB vs RAPV Hayabusas 2017 13:0
    Brook UFB vs Brook super converter (ver 2.2.2) with RAPVX 15:0
    Brook UFB vs MCZ TE2+ 16:0
    Brook UFB vs MCZ TE2 19:0
    Brook UFB vs GGXRD Custom fighter's pad 19:0
    Brook UFB vs RAPV Hayabusa 24:0


    Numbers refer to how many times one stick won versus the other, normally over 100 trials.



  • DarksakulDarksakul Your lack of faith disturbs me Joined: Posts: 22,046
    edited February 2
    noodalls wrote: »
    Nikogel360 has been very busy on youtube producing comparisons between different sticks for a few years. I thought I'd try to compile that work in one post.
    https //twitter com/nikogel360
    https //www youtube com/user/nikogel360/

    https //www youtube com/watch?v=sP0oINHdtl4
    Testing with Ryu vs Ryu (square buttons wired together) on PS3 Ultra SF4.
    RAPV4 (PS3 mode) vs PSU2TE2 (PS3 mode) 3:2
    RAPV4 (PS4 mode) vs PSU2TE2 (PS3 mode) 10:0
    RAPV4 (PS4 mode) vs PSU2TE2 (PS4 mode) 1:3
    RAPV4 (PS3 mode) vs PSU2TE2 (PS4 mode) 0:20
    RAPV4 (PS4 mode) vs RAPV3-SA 4:6

    https //www youtube com/watch?v=w1_xZoXRlo8
    Testing with PC USF4
    RAPVXSA vs RAPV silent 隼 (PS3 mode) 93:0

    (more later)

    https //www openrec tv/movie/jPgYoNkL4AL
    PS4 SFV
    Testing with Chun Li vs Chun Li
    Brook UFB vs XCM Cross fire converter (ver 3.0) with RAPVX 12:0
    Brook UFB vs RAPV Hayabusas 2017 13:0
    Brook UFB vs Brook super converter (ver 2.2.2) with RAPVX 15:0
    Brook UFB vs MCZ TE2+ 16:0
    Brook UFB vs MCZ TE2 19:0
    Brook UFB vs GGXRD Custom fighter's pad 19:0
    Brook UFB vs RAPV Hayabusa 24:0


    Numbers refer to how many times one stick won versus the other, normally over 100 trials.



    At least these results don't try to equate millisecond scores with a testing method that don't allow for actual timing like Teyah did. :coffee:
    "You must defeat Sheng Long to stand a chance."
  • ShinMagusShinMagus Joined: Posts: 395 ✭✭✭✭✭ OG
    edited February 2
    If we had the number of ties, we could calculate relative timing differences...
    Post edited by ShinMagus on
    Magus
    12000 B.C. - Dark Ages
    "The black wind begins to blow..."
    "Can you hear that? It's the sound of the Reaper."
  • handa711handa711 Joined: Posts: 142
    Any test for the new PS4 Slim/Pro controller using wired connection?
  • ShinMagusShinMagus Joined: Posts: 395 ✭✭✭✭✭ OG
    edited February 3
    I wasn't in any hurry to post this because I'll still go visit my friend who has the button with two pairs of quick disconnects soldered for additional tests (e.g., I want to put the PS360+ in PS3 mode in the mix) again at some point, but since I don't know when that will be, I decided to release what I got. He has a Razer Atrox and an LPT switch (it's supposed to be particularly fast), and I took a Hori Real Arcade Pro VX SA Kai (Xbox360) and a controller equipped with a Brook Universal Fighting Board (latest firmware) there. Here are our findings (we used Ryu's Solar Plexus Strike (f + hp) in USF4 (PC), each set consisting of 500 attempts):
    * Razer Atrox (Xbox360) vs Hori Real Pro Arcade Pro VX SA Kai (Xbox360)
    Atrox: 22
    Tie: 455
    HRAP VX SA Kai: 23
    
    Technical tie, given the methodology. The data would mean that the HRAP VX SA Kai is 0.0334ms faster, but I believe this kind of accuracy isn't really possible over 500 attempts (e.g., while we were testing and counting, sometimes one PCB would even take a "two-point" lead).
    
    * LPT switch vs Hori Real Pro Arcade Pro VX SA Kai (Xbox360)
    LPT: 94
    Tie: 404
    HRAP VX SA Kai: 2
    
    Convincing victory for the LPT switch. The data says the LPT switch is 3.0667ms faster. ((94 - 2) / 500 * 16.6667)
    
    * Razer Atrox (Xbox360) vs Brook Universal Fighting Board (XboxOne mode)
    Atrox: 17
    Tie: 436
    Brook UFB XboxOne: 47
    
    Brook UFB in XboxOne mode wins by 1ms. ((47 - 17) / 500 * 16.6667)
    
    * Razer Atrox (Xbox360) vs Brook Universal Fighting Board (Xbox360 mode)
    Atrox: 32
    Tie: 436
    Brook UFB Xbox360: 32
    
    Tie. This makes me wonder if these PCBs are at the limit of responsiveness for Xbox360 controllers.
    
    * Razer Atrox (Xbox360) vs Brook UFB (PS4 mode)
    Atrox: 0
    Tie: 374
    Brook UFB PS4: 126
    
    Impressive and unexpected victory for the Brook UFB in PS4 mode, given that the Atrox PCB is supposedly very good as far as Xbox360 controller PCBs go. 4.2-ms difference. (126 / 500 * 16.6667)
    
    * Razer Atrox (Xbox360) vs Brook UFB (PS3 mode)
    Atrox: 0
    Tie: 386
    Brook UFB PS3: 114
    
    Another big victory for the Brook UFB, this time in PS3 mode. 3.8-ms difference. (114 / 500 * 16.6667)
    
    * LPT switch vs Brook UFB (PS4 mode)
    LPT: 8
    Tie: 448
    Brook UFB PS4: 44
    
    Brook UFB in PS4 mode wins by 1.2ms. ((44 - 8) / 500 * 16.6667)
    

    Given what I've seen so far, including from tests posted by others (e.g., on this thread :)), I have a feeling the Brook UFB in PS4 mode is the fastest PCB out there. In XboxOne mode (XInput support) it's certainly excellent already, besting those Xbox360 PCBs like that. Also, the fact that in Xbox360 mode it's as good as them means we have three PCBs of different origins with the same performance and this may be a limit for Xbox360 controller PCBs.
    Post edited by ShinMagus on
    Magus
    12000 B.C. - Dark Ages
    "The black wind begins to blow..."
    "Can you hear that? It's the sound of the Reaper."
  • noodallsnoodalls Joined: Posts: 404
    ShinMagus wrote: »
    If we had the number of ties, we could calculate relative timing differences...

    100 trials each, so number of ties is anything where one doesn't win.
  • noodallsnoodalls Joined: Posts: 404
    handa711 wrote: »
    Any test for the new PS4 Slim/Pro controller using wired connection?

    I don't think he's done it. I have and posted the results on this very page.

  • ShinMagusShinMagus Joined: Posts: 395 ✭✭✭✭✭ OG
    edited February 2
    noodalls wrote: »
    ShinMagus wrote: »
    If we had the number of ties, we could calculate relative timing differences...

    100 trials each, so number of ties is anything where one doesn't win.

    Oh, ok. :)

    RAPV4 (PS3 mode) vs PSU2TE2 (PS3 mode) 3:2 95 ties imply the data says there's a 0.16ms difference, but I don't think 100 attempts can provide this kind of accuracy, so if it's 3-2, it's probably a tie.
    RAPV4 (PS4 mode) vs PSU2TE2 (PS3 mode) 10:0 90 ties imply the data says there's a 1.67ms difference.
    RAPV4 (PS4 mode) vs PSU2TE2 (PS4 mode) 1:3 96 ties imply the data says there's a 0.33ms difference. This might actually be a tie, since 1-3 is very close over 100 attempts...
    RAPV4 (PS3 mode) vs PSU2TE2 (PS4 mode) 0:20 80 ties imply the data says there's a 3.33ms difference.
    RAPV4 (PS4 mode) vs RAPV3-SA 4:6 90 ties imply the data says there's a 0.33ms difference. This might actually be a tie, since 4-6 is very close over 100 attempts...

    Testing with PC USF4
    RAPVXSA vs RAPV silent 隼 (PS3 mode) 93:0 7 ties imply the data says there's a 15.5ms difference.

    https://www.openrec.tv/movie/jPgYoNkL4AL
    PS4 SFV
    Testing with Chun Li vs Chun Li
    Brook UFB vs XCM Cross fire converter (ver 3.0) with RAPVX 12:0 88 ties imply the data says there's a 2ms difference.
    Brook UFB vs RAPV Hayabusas 2017 13:0 87 ties imply the data says there's a 2.17ms difference.
    Brook UFB vs Brook super converter (ver 2.2.2) with RAPVX 15:0 85 ties imply the data says there's a 2.5ms difference.
    Brook UFB vs MCZ TE2+ 16:0 84 ties imply the data says there's a 2.67ms difference.
    Brook UFB vs MCZ TE2 19:0 81 ties imply the data says there's a 3.17ms difference.
    Brook UFB vs GGXRD Custom fighter's pad 19:0 81 ties imply the data says there's a 3.17ms difference.
    Brook UFB vs RAPV Hayabusa 24:0 76 ties imply the data says there's a 4ms difference.

    I'm skeptical about numerical accuracy over 100 trials... Even 500 trials make me a little uneasy. I don't know how many trials would be needed for 0.01ms accuracy or what the margin of error is for 100 and 500 trials.
    Post edited by ShinMagus on
    Magus
    12000 B.C. - Dark Ages
    "The black wind begins to blow..."
    "Can you hear that? It's the sound of the Reaper."
  • DarksakulDarksakul Your lack of faith disturbs me Joined: Posts: 22,046
    ShinMagus wrote: »
    If we had the number of ties, we could calculate relative timing differences...

    Stop, stop, just stop. That testing methodology does not allow for millisecond timings.
    All you can do is comparative results.

    Assigning any timed results from VS testing is a fallacy. You are asking for a metric the testing never took into consideration.
    With VS testing you also have to factor in a controller interfering with the results of the other.
    ShinMagus wrote: »
    I don't know how many trials would be needed for 0.01ms accuracy or what the margin of error is for 100 and 500 trials.
    You don't. This kind of testing does not and will not give ms results. All that would happen with more testing cycles is you have more accurate VS results.
    All you have is what one controller does better than the other on average, not their latency in ms. You can test a billion times with each controller and never have accurate ms results as time was never a metric that was measured. All that being tested for is Win, Lost and Tied, that is it.
    "You must defeat Sheng Long to stand a chance."
Sign In or Register to comment.