3rd Strike Lag/Speed Tests

13

Comments

  • alien nose job.alien nose job. Joined: Posts: 88
    edited June 2014
    Hey there, congratulations for your persistence at running these tests, it is a good thing that someones tries to sort things out and get some facts !

    I discovered shmupmame in your thread, and I was quite surprised...
    Since you also mentioned Groovymame, I had a good look at the forum where the devs post.

    Here is what I understood, roughly :

    - Shmupmame is very easy to use, and reduces input lag by breaking the process of emulation, which can result in, mainly, visual inaccuracies (but so far I couldn't spot anything except on random screenshots). The other downside being that you can't use Vsync or tripple buffering if you need it to get rid of screen tearing... well, not completely sure of that part.
    It also offers a nice way of mimicking the arcade looks using HLSL.
    I fiddled with it for a few weeks now, trying to check it out a bit after spending some time playing at the arcade. But I don't know if it does add lag or not. (it may not be the case as the various hacks of this emulator just cut the cue of rendering to get straight to business, so this may be something to test).
    I'll try to edit with screenshots later on, even though my HLSL settings are not there yet.

    - GroovyMame changes the way emulators work, and the specific order in which they do run the steps of emulating, making it accurate, but built in a more logical way, perfomance wise.

    Bottom line, so far, both emulators seem to get the same results regarding frames of delay.
    But, GroovyMame devs seem very responsive and striving to get to the point where you won't have any lag coming from the emulator (while flagging the outside causes and trying to find solutions) while remaining accurate. Great project. But seems like a pain in the ass to use for now. so we can hope that at some point they'll get a guy and more things to make it user friendly.

    So right now I'm using Shmupmame, and even though I do have a laggy TV, it's getting damn close to the arcade, IMAHO.

    Hope some of this makes sense.

    I'll add links to threads worth reading if you want to check the detail of what I tried to explain there, later on.

    [EDIT] :::

    - Here a link to Papasi's thread here on SRK where he made all the same lag tests and more, specifically on ST using his PCB and Shmupmame as comparison.

    - Somwhere along the thread, Jdubs came in and linked to a thread where he started to talk to GroovyMame's devs about input lag. They've been very responsive and tested a lot of thing and actually made some great discoveries that should largely improve their next release. Very long but interesting read. btw, I'd have to check again to confirm before spreading bad info, Seems like in this thread, Dark Gaiden has tested the influence of bilinear filtering and claims it adds some lag. But I can't say if he tested it or if it's a hunch, I'll message him to see and edit later. Hence me posting screenshots below with and without this option turned on as it's fairly common to use it to make the game lookable on a LCD (not needed at all with HLSL)

    [EDIT 2] :::

    - here is also a link to the official ShmupMame page, it is pretty short to read and understand the means the dev uses to get it going.
    Post edited by alien nose job. on
  • ilitiritilitirit Joined: Posts: 6,381
    Can you also please post which input methods (and the brands/manufacturers) you tested? In my experience the MadCatz TE PCBs are slightly laggier than some other brands. I don't know if they're universally laggier across all platforms or just on PS3 though.

    It's pointless to argue over 1f of monitor/emulator input lag if your controller is just as laggy.
  • alien nose job.alien nose job. Joined: Posts: 88
    Sure, I used both an old crappy Hori Ex2, and a HRAP VX SA for Xbox360.

    Using USB sticks on emulator seems to add 1 frame of lag, from what I read it should use keyboard material or some sort of bridge to get rid of it, but this is not my point. But it seems that the guy that works on GroovyMame is aware of the issue, and will ask official mamedevs to change it in future iterations to make implement RAW input for sticks too, not just keyboard.

    I just meant to say Shmupmame (and most likely GroovyMame) are an improvement on FBA and old Mame.

    Don't get me wrong, I don't mean to fuel a flame war of what's good what's not, like PC sucks or rocks.
    I just think the more access we have to better versions of the game, the better.
    And it seems like something cool aside console, which is good in other ways but also sucks.

    As promised, here are screenshot comparisons of different graphic options on Schmupmame (or any emu right now I think) to display HLSL:

    TESTHLSLCUTbigger01_zpse04cd48c.pngTESTHLSLCUTbigger02_zpsa5d1983d.png

    Sorry for the big image, spoiler tags don't seem to work.

    Please bear in mind I edited the HLSL settings more or less as I could, there's slightly too much blue and red, I have to get back to the arcade to check again.
    Still, it does look a lot more like arcade, and way better than the regular scnalines filter, which is great if you're stuck with a LCD.

    By the way, I mentionned Bilinear filtering as it seems that it also adds lag. Yet I have to check again to confirm.
  • ilitiritilitirit Joined: Posts: 6,381
    I was actually referring to the tests performed by the OP, but thanks anyway.
  • isotopezisotopez Joined: Posts: 125
    ilitirit wrote: »
    Can you also please post which input methods (and the brands/manufacturers) you tested? In my experience the MadCatz TE PCBs are slightly laggier than some other brands. I don't know if they're universally laggier across all platforms or just on PS3 though.

    It's pointless to argue over 1f of monitor/emulator input lag if your controller is just as laggy.

    Not sure what you mean by "input method," but I listed almost everything I did in the original post. 360 was on hori pcb. Ps3 and emulator on mc Cthulhu.
  • ilitiritilitirit Joined: Posts: 6,381
    Weird. For some reason my browser caches page 2 as page 1, so I only see the first post from page 2, not the original that listed the controllers. Anyway it's sorted out now. I had to clear my browser cache and log in again.
  • Lance3rdLance3rd Alaska 3rd Strike Joined: Posts: 986
    bk2099 wrote: »
    isotopez wrote: »
    I've finished rerecording and recounting footage for the 360 and dreamcast. I decided to only count the frames at 120fps and then divide those numbers by 2, which gives the lag at 60fps with half a frame of precision.

    The videos for the consoles aren't consistent with cps3 because they refresh at a different rate, so the console averages are likely skewed a little compared to cps3.

    These numbers do not mean that 360 lags more than the ps3 or even cps3. Both consoles' averages fall within the same range of error.

    3S Lag Results:
    CPS3 - 3.6
    PS2 - 4.7
    PS3 (filters off) - 3.8
    PS3 (filters on) - 3.8
    360 (filters off) - 4.1
    360 (training mode) - 3.9
    Dreamcast - 6.1

    *All these averages should be +/- 0.5 frames of the actual lag.




    I was wondering if you could retest the Dreamcast version when you get some time. I would do it myself but I don't have the recording gear.

    The reason is because the CPS3 hardware uses a Hitachi SH-2 as its CPU and the Dreamcast uses the Hitachi SH-4, which is in the same processor core family. This makes the Dreamcast the closest to the CPS3 in terms of hardware.

    Also if you could perform the tests:
    -in 240p mode [Hold Start+L (or Start+Z on the Arcade Stick) before the Capcom logo appears to get the 320x240 mode.]
    -using a "Madcatz Dream Pad" or any other 3rd party controller since the stock Dreamcast controllers are known to lag.
    -using the Street.Fighter.III.Third.Strike.Arcade.Edition-RDC ISO. Apparently it's arcade perfect with unblockables.
    -bonus if you can wire it up to an arcade monitor in 15khz RGB mode.

    I'm not sure what isotopez' thoughts on this are - from my perspective, I think we could do it but it would be really low priority. it's taken a good while to actually set up the speed tests (plus we both work so it hasn't always been our top priority lol) and we plan on actually doing those pretty soon. we've thought about going back and re-testing for input lag with isotopez' more accurate method after we're done with the speed stuff, and we still might do that.

    IMO Dreamcast and PS2 are kinda bottom of the importance list. there's more to be gained from testing the big platforms that a lot of people use - CPS3, OE, and emulator. not saying we won't do it, just won't be at the top of the do list.
  • isotopezisotopez Joined: Posts: 125
    I doubt we'll do anymore tests with the dreamcast. There was 1 thing I still need to find out for esn, but other than that, it seems bad enough that no one should be playing 3s on it. I don't have access to a ps2 anymore, so we probably won't run speed tests on it or retest the lag with a stock controller either.
  • TiredOceanTiredOcean Low-tier theory fighter Joined: Posts: 134
    Interesting find isotopez. Although I can't say I understand all of this, would the main difference in speed between the console and CPS3 versions be due to the difference in refresh rates, or would you put it down to how 3rd Strike is emulated in OE?
    Some simple maths shows that, if we assume the refresh rates are the only difference, CPS3 3S runs at ([59.583*100]/60 =) 99.305 % the speed of OE. If only there was a way to hack the game to slow it down by this amount...
    I've tried running shmupmame at 0.99305 speed but it still feels too fast, but I haven't tried any in-depth tests.
  • Lance3rdLance3rd Alaska 3rd Strike Joined: Posts: 986
    I think it means the perceived difference in gameplay speed is just down to refresh rate. I expected going in that we'd find a smoking gun like "you can cancel Chun's low forward into super on CPS3 much later than on OE" since it definitely feels that way, but it didn't seem so. you can tell from how the tight links dropped on OE that there's definitely a timing difference, and yeah for anything that requires tight timing several times in a row it'll eventually drop.

    so I think it's fair to say what people have always said - OE is not the same as CPS3 but it's reasonably close. if you play characters that require tight links you'll probably notice the difference. if you don't, you'll probably have a slight bit more time to react to things on CPS3 but it's not huge.

    we haven't tested audio latency at all, and that could be a factor too. if OE had a longer audio buffer you could be hearing your hit sounds later and would have less time to confirm if you confirm off sound. it's definitely possible. any stage or move that causes slowdown on CPS3 will make a difference as well. and of course if you're playing on a laggy stick or monitor or playing online, things will be really different (goes without saying).

    also regardless of our results, OE's sound effects still suck. there's no cure for Oro snoring and trashcan shoryus.
  • isotopezisotopez Joined: Posts: 125
    edited June 2014
    The games do run at different speeds, so if you're a robot with a super accurate ability to push buttons at specific times, you might notice the difference in speed if you try doing long combos like elbow cannons, tth on the twins, or some very timing specific genei jin combos. Whatever combo you're doing though has to last long enough for your inputs to desync with the game speed, like tebbo was saying. That of course only matters if you're used to playing on 1 platform and you switch to the other. Most people probably can't do x6 elbow cannons on urien consistently without double tapping buttons. If they can though, they have visual and audio feedback that they probably depend on more so than an internal clock in their head. I can't say that is true about everyone, but for most people it is.

    I think there may also be a perceived difference in timing just based on how easy it is to get your input to come out on the frame you intended it to come out on between cps3 and oe. Perhaps USB on 360 is prone to occasional lag, which would change the timing of your inputs. This would only become apparent if you press your button on the very edge of cancel or link windows i.e if the move comes out 1 frame late you drop your combo.
    Post edited by isotopez on
  • yuukiyuuki Joined: Posts: 782
    "The games do run at different speeds"
    "'May be a a perceived difference in timing"

    If the game runs at a different speed then it's not a perceived difference in timing but actually a difference in timing
    かかってきな。
  • isotopezisotopez Joined: Posts: 125
    I added an "also" to my post (May also be a perceived...). I see them as 2 different reasons people will say there's a timing difference. The actual timing difference is very minute whereas a perceived difference isn't really a timing difference, but it'll feel like one and it would probably be more noticeable.
  • TebboTebbo Play. Joined: Posts: 5,688
    that plus any input lag differences make it easy to notice/feel.
    it's possible to sense something which measurements might tell us is essentially negligible. humans are remarkably acute.
    Play more.
  • TiredOceanTiredOcean Low-tier theory fighter Joined: Posts: 134
    Well...... A lot of audiophiles claim they can "sense" differences between music files/sound systems that they really can't (i.e. the difference between a 320kbps MP3 vs. lossless audio), which lead them to spending thousands on equipment that don't really make a difference.
    But that's perceiving differences in sound quality, while we're looking for differences in speed. Humans might be a bit better at that.
  • TebboTebbo Play. Joined: Posts: 5,688
    so you're saying without knowing already that we couldn't tell the difference?

    we might exaggerate sure. but the game feels different. 360 OE on a crt is pretty good though.
    not everyone might notice, but some probably do 100% of the time. there is 3S and then there are lesser versions of 3S.
    even if its small, differences exist and they are noticeable. they do add up. it's immediately apparent in my experience. super links are an easy example.

    so like isotopez is saying. it might be pretty small, but the actual effect of say 1 frame of input lag more, can be pretty enormous (perceived). especially if the gameplay is running faster and the inputs are running slower (like the ports). so in the end you just have that much less time to react to something. teching throws is easier on cps3, period.
    Play more.
  • DanderDander BANNED Joined: Posts: 7,227
    sharp in contrast deez nuts
    8tROOXi.png
  • Lance3rdLance3rd Alaska 3rd Strike Joined: Posts: 986
    http://forums.shoryuken.com/discussion/191312/how-much-does-your-arcade-stick-lag-arcade-stick-input-lag-testing-results-thread

    just wanted to post this here. it's a rundown of various arcade sticks and their innate input lag. as you can see every arcade stick varies. just one more thing that could add up to a different experience. for instance when OE first came out there was a general feeling that OE lagged more on PS3 than 360, but this could just as easily be arcade stick as anything else. even a stick that is modded for several different consoles can have different delay on each platform! or a stick with no input delay on console might have poorly written drivers for PC and have delay there.

    every individual component in the chain adds up to the overall experience.
  • ryan.ryan. Joined: Posts: 1,918
    Holy shit well that might explain why I thought ps3 was shittier than Xbox. I've only tried ps3 at last years Evo where I played about 40-50 matches on a standard TE and it felt like complete dog shit. I blamed the poor ps3 never even thinking to blame the true turd, the TE.

    That also might explain why I've never felt comfortable on a TE on Xbox as well. About a half frame lag.

    Now I guess really it doesn't matter if you ONLY use your stick on console. But if you are borrowing a stick or primarily play arcade, this is pretty significant.

    Ps3 TE omg
  • TebboTebbo Play. Joined: Posts: 5,688
    edited July 2014
    neat info.
    16.66ish ms is about a frame (assuming around 1/60 of a second).
    looks like i should invest in an etokki omni lol
    Play more.
  • NoMoreFunlandNoMoreFunland Perfection equals obliteration! Joined: Posts: 1,229
    edited July 2014
    You sold me a bunk component Lance! Lol

    Real shame too. I wanted to get a PS3 sooner than later and try some of the PSN comp. Could buy a vx-sa.

    All these tests are great. So much theory flying around about lag early in the OE run. Amazing how people shut their brains down when they've decided a theory is the truth. Like people saying less than 1 frame of lag is imperceptible even though many people notice the difference.

    I wish MadCatz was paying attention to these things when they started making expensive joysticks.
    Post edited by NoMoreFunland on
  • Lance3rdLance3rd Alaska 3rd Strike Joined: Posts: 986
    It's hard for companies to pay attention when you have a good chunk of people in threads like that claiming 1-2 frames of lag doesn't matter or it's impossible to tell the difference. What a joke.

    Really discouraging how much dumb you have to weed through whenever you test stuff like this. Can speak from personal experience lol, it really saps your energy and interest. At least for us it was only one person, that guy is dealing with a large group of them. You set out to get to the bottom of something and learn more, and along the way you have to defend yourself from people who think the whole thing is imaginary and aren't interested in testing anything themselves but are 100% interested in voicing their opinion.
  • TebboTebbo Play. Joined: Posts: 5,688
    edited July 2014
    yeah there are some straight up dumb people is what i learned.
    or maybe just it's clear who is coming from a 3S perspective and who isn't. like in sf4 where the difference of a single frame matters a lot less because all the hit confirms are multiple normal sequences with static timing which give you a huge window to see if things are hitting.
    Play more.
  • petran79petran79 Here comes an old challenger Joined: Posts: 1,913
    sorry for not having knowledge in the matter, but one question:

    do better or improved input drivers compensate for input lag?

    Eg if you compare Shmupmame, released 18 months ago, to the latest MAME, a lot of changes were made to drivers, input included.
    If controllers can feel more responding and inputs are registered with greater accuracy, like in the arcade version, isnt this better than small input lag yet inaccurate inputs and timing?


    too slow!
  • DanderDander BANNED Joined: Posts: 7,227
    Yes there can be an increase in performance through the use of improved input drivers. I recall using a booster for my usb ports on my last rig and suddenly finding ggpo to be playable. That changed soon thereafter but the change was definitely noticeable.
    8tROOXi.png
  • isotopezisotopez Joined: Posts: 125
    If the improved drivers do in fact make the inputs more consistent, then it's a matter of whether or not you have more fun playing in persistent lag vs having your inputs occasionally lag. It's also possible that the variance in the lag data for the emulators is caused by inaccurate measurements from the monitor ghosting (we didn't have access to a CRT).
  • free_manfree_man Joined: Posts: 81
    If someone is in Japan and can play the Nesica version of 3S, please, can tell to us - after test it, of course - how is the speed of this system in comparison the others?
  • SpoRSpoR SpoR Joined: Posts: 350
    So by comparison, how does it run on GGPO, Supercade, and fightcade etc>
    Zerg!
  • cribbacribba Joined: Posts: 22
    Does anyone know if the default theme in Windows 8.1 causes lag like Aero in 7?

    If you play in fullscreen does the theme still cause lag?
  • exodusexodus anti-vid scrub advocate! Joined: Posts: 2,078 ✭✭✭✭✭ OG
    Windows 8 has aero permanently on.
    .
  • lamhazaartlamhazaart Joined: Posts: 91
    exodus wrote: »
    Windows 8 has aero permanently on.

    You can disable Aero in Windows 8, but it's a mildly involved process. You have to suspend winlogon.exe and then kill dwm.exe using procexp. You also have to rename some system dll files the first time you do it.
  • lamhazaartlamhazaart Joined: Posts: 91
    edited July 2015
    Regarding game speed, it turns out it's very easy to find out PC emulator framerate. Just need fraps+frametime viewer. Fraps' display already shows 59.5fps (only shows 3 decimal places) in speed-accurate emulators like Shmupmame and Groovymame. Looking at frame times, you see a frame interval correlating to that speed. The emulators themselves also indicate that they're running at 59.583. There's no need for adjustment for these emus.

    I found this out when testing black frame insertion in Groovymame, which can only run at exact monitor refresh rate (so, in my case 120hz). This caused Fraps to show 120fps, 8.33ms frame intervals (1/2 of 16.66), and an increase in speed when I timed single and multiple rounds equivalent to the difference in speed between 59.583fps and 60fps.

    TL;DR Shmupmame and Groovymame are already speed accurate without adjustment. However, my personal recommendation is to get them running at 60fps if you're on an LCD monitor - this means video will be smooth, frame delay will be consistent, and the current tournament standard(?) is 3SOE which runs at 60fps. If you're on a 120hz monitor, use black frame insertion to get near-crt motion smoothness.
    Post edited by lamhazaart on
  • IbokiIboki Joined: Posts: 206
    All things considered would it be better to practice offline on 360 OE or the ntsc-j PS2 port. I have ps2 pad/stick I'm comfortable with, and an xtokki converter for 360. I know ps2 speed is way off, but I find the xtokki converter to be pretty bad, even though it was hyped up by Markman and stuff. Once I can get my hands on a 360 stick it'll be OE for sure.
  • isotopezisotopez Joined: Posts: 125
    edited July 2015
    Ps2's speed isn't any different than oe's since both run at 60fps. Our tests showed ps2 being laggier than oe, but we aren't sure if that's because of controller lag or not. Just practice offline on whatever you have unless it's dreamcast.
  • IbokiIboki Joined: Posts: 206
    cool thanks!
  • shanoshano Joined: Posts: 284
    Doesn't seem like we can turn off Aero on Windows 10 from what i have gathered? Able to do a test on FBA and see the input delay results Windows 10 gives?
  • akuaku スピリチュアルやからね! Joined: Posts: 3,865
    Tried out Fightcade on Windows 10. Honestly I couldn't FEEL any additional lag/delay on it, but I'll leave it to someone else to explain if there actually is. :v
  • ChadChad Joined: Posts: 1,291
    Have you tried Fightcade on 8.1 aku? I played for a few hours and it was intolerable. Everything was underwater.

    Is 10 better?
  • isotopezisotopez Joined: Posts: 125
    shano wrote: »
    Doesn't seem like we can turn off Aero on Windows 10 from what i have gathered? Able to do a test on FBA and see the input delay results Windows 10 gives?

    We'll test it
  • akuaku スピリチュアルやからね! Joined: Posts: 3,865
    Chad wrote: »
    Have you tried Fightcade on 8.1 aku? I played for a few hours and it was intolerable. Everything was underwater.

    Is 10 better?

    Definitely felt better than 8.1, that thing felt like playing on a crappy Dell monitor. FBA was fine, Fightcade was Fightcade. I'm playing on a subpar monitor so there's always gonna be some delay on my part, but I never felt any additional delay in comparison.
  • JedahsMinistryJedahsMinistry BattousaiCV Joined: Posts: 542
    How about frameskipping/stuttering?
    Gettysburg, PA
    Netplay Poverty - http://www.twitch.tv/randomalt
    X-Men COTA and stuff - http://www.youtube.com/user/BattousaiCV
    Blurbs about games - http://twitter.com/BattousaiCV
  • vandarkholmevandarkholme Joined: Posts: 388
    I heard that different settings in 3soe bring out different results. Is that true?
    I'm planning to run a tournament with it and I'm not sure if I should worry about it
    Sutorito Faita : Sad strike
    vandark, you should hear a cool pherai story from me one day then. Add it to all your history. I'll probably remember it closer to EVO time and just retell it to the forum when I'm probably going to re-tell some more anyway. Its not the glory hole one though, that one's too legendary. You probably know more about that than me.
  • Smashbro29Smashbro29 Waiting for the new Framemeister... Joined: Posts: 7,185
    lamhazaart wrote: »
    exodus wrote: »
    Windows 8 has aero permanently on.

    You can disable Aero in Windows 8, but it's a mildly involved process. You have to suspend winlogon.exe and then kill dwm.exe using procexp. You also have to rename some system dll files the first time you do it.

    Tutorial please.
  • shanoshano Joined: Posts: 284
    isotopez wrote: »
    shano wrote: »
    Doesn't seem like we can turn off Aero on Windows 10 from what i have gathered? Able to do a test on FBA and see the input delay results Windows 10 gives?

    We'll test it

    Just curious if this is still happening?
  • isotopezisotopez Joined: Posts: 125
    shano wrote: »
    Just curious if this is still happening?
    We had camera issues. It's looking like we won't be able to test now
  • DontdodatDontdodat I know u want to Joined: Posts: 653
    Damn, I was going to run 3S meets on 360 (OE) CRT set ups. Probably still will because there are no cabs I can use.
  • DontdodatDontdodat I know u want to Joined: Posts: 653
    I don't think we have any Yuns good enough to make a diference with command grab. This is all a stepping stone to supergun and maybe a cabinet one day.
Sign In or Register to comment.