3rd Strike Lag/Speed Tests

135

Comments

  • jblairjblair Joined: Posts: 1,435
    I thought it was known that emu 3s is no good? I doubt that a better pc makes it perfect. It's way faster than cps3
  • mpikkonmpikkon "S" class tier (OP) Joined: Posts: 355
    edited April 2014
    ryan. wrote: »
    I was just wondering the price. If you're saying 3rd Strike is perfect emulated but only if you spend $1000+ then there's really no point to discuss it. I can just buy a cps3 myself.
    We don't have the official PC release, am I correct? It should have been released. An official PC version vs console versions.
    ryan. wrote: »
    Also, your tutorial is impressive. I can tell you put in a lot of time and effort. Good job buddy. But I was asking if you actually have hands on experience with cps3 3rd Strike. If you only have limited time then it's hard to take your word when you say emulated 3rd Strike is good.
    I have no local 3s scene, nor enough free time to watch 3s videos or play on GGPO as I used to do. Still, my contribution is highly appreciated. Thank you, ryan.
    Post edited by mpikkon on
  • yomipoweryomipower not a legendary game designer Joined: Posts: 1,167
    You can be autistic about it and buy a $1500 Alienware PC if you wanna be a retard, but it doesn't change the fact that CPS3 emulation is imperfect across all emulators due to them using the same codework pretty much.

    Also Lance/isotopez/whoever, can you test the first CPS3 emulator by Nebula? I'm sure everyone has forgotten about it by now since it doesn't have netplay and it's ancient, but just for the sake of completion. You can get the emu from Nebula's homepage, but IIRC the rom needs some custom BIOS file or something in the .zip or it wont work, regular sfiii3 romsets from the usual sites didn't work.
    Yomi, which is the Japanese word for the underworld. Also a brand of vitamins for children.
  • Lance3rdLance3rd Alaska 3rd Strike Joined: Posts: 986
    yomipower wrote: »
    You can be autistic about it and buy a $1500 Alienware PC if you wanna be a retard, but it doesn't change the fact that CPS3 emulation is imperfect across all emulators due to them using the same codework pretty much.

    Also Lance/isotopez/whoever, can you test the first CPS3 emulator by Nebula? I'm sure everyone has forgotten about it by now since it doesn't have netplay and it's ancient, but just for the sake of completion. You can get the emu from Nebula's homepage, but IIRC the rom needs some custom BIOS file or something in the .zip or it wont work, regular sfiii3 romsets from the usual sites didn't work.

    Sure thing. It'll be when we swing back around to the lag tests, but we can do that.
  • exodusexodus anti-vid scrub advocate! Joined: Posts: 2,078 ✭✭✭✭✭ OG
    As much as this is helpful in fueling our understanding of the intricate differences of each platform, I am fairly certain it is at least to some extent counterproductive in promoting the health of 3S as a whole. Too often do I notice people coming out in favor of one or the other, or neglecting to participate when given the option, simply because it is in their minds an inferior/incorrect version of 3S. I can say with 100% confidence that close to none of these people (or their respective competition) are at a point in their game where they are good/proficient enough in every area of the game to forego playing it in any inferior iteration (minus DC). I commend what isotopez/igb/anyone else for what they do in order to increase our understanding of the nuances surrounding this magnificent fighter, but I think it needs to be said that you should promote the playing of 3S wherever you can get it.
    .
  • exodusexodus anti-vid scrub advocate! Joined: Posts: 2,078 ✭✭✭✭✭ OG
    edited April 2014
    I'm not saying what you're doing serves no purpose. It's definitely a very good thing; moreover, I wanted to state that regardless of the results, most of everyone should not avoid a certain platform because it is "laggier". Dreamcast stands an exception because it shifts the startup and active frames, resulting in a literal change in the game.
    .
  • d3vd3v Coughing DAT PINK SPIT Joined: Posts: 36,723 mod
    @mpikkon‌
    Flag abuse is a bannable offense. Also, your blatant PC fanboy-ism has added nothing of objective value to the thread, especially in the face of what we already know (all LCDs have add input delay over CRTs, 3S PC emulation being imperfect, etc.).
  • General_AwesomoGeneral_Awesomo "Hyouh!" Joined: Posts: 370
    Something here is making me scratch my head. There's PS3 results for filters on and 360 results for training mode. Why is there no PS3 results for training mode and 360 results for filters on? Did you guys just forget or something? And why even bother with training mode in the first place?
    GT: GeneraI Awesomo USFIV Main: Vega
    PSN: General_Awesomo UMvC3 Team: Zero (β) / Taskmaster (α) / Vergil (γ)
  • Lance3rdLance3rd Alaska 3rd Strike Joined: Posts: 986
    Ah yeah I guess that wouldn't make sense on first glance lol.

    We tested ps3 first, then 360 after the recent patch for both consoles. We tested filters on and off with ps3 and concluded from that that filters don't add any lag on OE. So we figured no point in testing filters on both consoles since it was likely the same. In between testing ps3 and 360 Ryan and a friend of his asked us to test OE training mode since they thought it might be adding lag. So we folded that into our 360 test. we probably would've tested filters and training mode on both consoles if there looked like there was any difference. but from the looks of things it's all the same between the two OE ports.

    we'll know if that's true for sure later when isotopez does his more accurate test. I believe the reason there's a difference in result numbers is because of where the inputs landed in the frame and where the camera frames landed in comparison to the monitor frames. for instance consider three possibilities of when you input a button:

    beginning of a frame - will show up 3 frames later, basically on time
    middle or towards the end of a frame - will show up 3 frames later, so same frame as if you did it at the beginning but less "real time"
    very end of a frame - will miss the window for making that frame, will actually show up 1 frame later, so you'll get what looks like a random extra frame of lag

    that's my understanding for why our results are not in whole numbers and why a bunch of platforms that presumably lag the same would have slightly different results (but all within the margin of error). I know you didn't ask, but thought I should explain in case it wasn't clear why I'd say 360 and PS3 are probably the same.
  • NoMoreFunlandNoMoreFunland Perfection equals obliteration! Joined: Posts: 1,229
    Exodus, GGPO is the most inconsistent platform for the game. That's just fact due to a million different PC setups even putting aside the inherent lag spoken of here.

    Lance and iso's research is good in many ways. One being that it snaps players out of the over simplified notion of "the arcade ROM is used therefore it's a superior online experience because its arcade perfect." With my 4 year old MacBook Pro I get noticeable input lag with delay set to 0 and even more sever audio lag. Ruins the learning for me but I haven't given up yet.

    Tbh your arguments seem more about bringing players into the GGPO environment rather than getting players to be less biased to all platforms. If anything this research should encourage GGPO players to buy a cheap console and play on both.
  • exodusexodus anti-vid scrub advocate! Joined: Posts: 2,078 ✭✭✭✭✭ OG
    edited April 2014
    You're mistaking my preference with what I am trying to say. I prefer GGPO. I said "Play 3S any which way you can get it, because the exposure is more beneficial than detrimental." Unless you are somehow assuming I am siding with mpikkon, which I was not. Their little back and forth is what made me post to begin with.
    .
  • NoMoreFunlandNoMoreFunland Perfection equals obliteration! Joined: Posts: 1,229
    edited April 2014
    Your clarification is more clear and concise, thanks. I hope you are helping your preferred community to see the validity of playing on platforms other than GGPO. Less bias is good for everyone.

    Edit: nice ninja edit exo. It was better in the first draft. Lol
    Post edited by NoMoreFunland on
  • WTF-AKUMA-HAXWTF-AKUMA-HAX DBGT non canon > maxxvatar OVA Joined: Posts: 17,937
    edited May 2014
    isotopez wrote: »
    Our next goal is to test for speed differences and input timing differences. My plan for the input timing tests is to make a programmable controller. We're thinking of testing necro's elbow cannon juggles on urien (1 frame link), cancelling normals into stuff with different cancel timings, and super input windows. Let me know if there's anything specific you guys want to know about, and we might be able to fit it in.
    Its a lot more tests to do and needing some baseline for comparison like those target combo or cancel ideas, and towards the last of the things to test maybe, after all the bigger stuff

    but is there something anyone has thought of to determine each stage's gamespeed? Of course you figure the ones with more going on in background objects or always on-screen layered effects like Yang's rain are the more "laggy" ones.

    Otherwise we get the offhand comments that "lol those guys looking for lag, someone found out every stage lags differently! So you are all wrong!" seen it before and no links to what they were talking about. Its not out of the ordinary that SF games have different stage lag, which the SFA2 people know directly as it changes things during Custom combos, and was pinpointed to background objects and the game accounting for those. Having them removed with edits, reduced or got rid of it all.
    1999 = "A Game with Parries isn't Street Fighter"
    2016 = "Releasing a Complete Game isn't Street Fighter"
    You wouldn't even understand if I told you.
    People will forget what you said.
    People will forget what you did.
    But, people will never forget how you made them feel.

  • bk2099bk2099 Joined: Posts: 99
    edited June 2014
    isotopez wrote: »
    I've finished rerecording and recounting footage for the 360 and dreamcast. I decided to only count the frames at 120fps and then divide those numbers by 2, which gives the lag at 60fps with half a frame of precision.

    The videos for the consoles aren't consistent with cps3 because they refresh at a different rate, so the console averages are likely skewed a little compared to cps3.

    These numbers do not mean that 360 lags more than the ps3 or even cps3. Both consoles' averages fall within the same range of error.

    3S Lag Results:
    CPS3 - 3.6
    PS2 - 4.7
    PS3 (filters off) - 3.8
    PS3 (filters on) - 3.8
    360 (filters off) - 4.1
    360 (training mode) - 3.9
    Dreamcast - 6.1

    *All these averages should be +/- 0.5 frames of the actual lag.




    I was wondering if you could retest the Dreamcast version when you get some time. I would do it myself but I don't have the recording gear.

    The reason is because the CPS3 hardware uses a Hitachi SH-2 as its CPU and the Dreamcast uses the Hitachi SH-4, which is in the same processor core family. This makes the Dreamcast the closest to the CPS3 in terms of hardware.

    Also if you could perform the tests:
    -in 240p mode [Hold Start+L (or Start+Z on the Arcade Stick) before the Capcom logo appears to get the 320x240 mode.]
    -using a "Madcatz Dream Pad" or any other 3rd party controller since the stock Dreamcast controllers are known to lag.
    -using the Street.Fighter.III.Third.Strike.Arcade.Edition-RDC ISO. Apparently it's arcade perfect with unblockables.
    -bonus if you can wire it up to an arcade monitor in 15khz RGB mode.
  • exodusexodus anti-vid scrub advocate! Joined: Posts: 2,078 ✭✭✭✭✭ OG
    No, seriously. DC is not 3S at all because there's a global shift in startup/active/recovery frames. It's the least accurate out of any iteration.
    .
  • alien nose job.alien nose job. Joined: Posts: 88
    edited June 2014
    Hey there, congratulations for your persistence at running these tests, it is a good thing that someones tries to sort things out and get some facts !

    I discovered shmupmame in your thread, and I was quite surprised...
    Since you also mentioned Groovymame, I had a good look at the forum where the devs post.

    Here is what I understood, roughly :

    - Shmupmame is very easy to use, and reduces input lag by breaking the process of emulation, which can result in, mainly, visual inaccuracies (but so far I couldn't spot anything except on random screenshots). The other downside being that you can't use Vsync or tripple buffering if you need it to get rid of screen tearing... well, not completely sure of that part.
    It also offers a nice way of mimicking the arcade looks using HLSL.
    I fiddled with it for a few weeks now, trying to check it out a bit after spending some time playing at the arcade. But I don't know if it does add lag or not. (it may not be the case as the various hacks of this emulator just cut the cue of rendering to get straight to business, so this may be something to test).
    I'll try to edit with screenshots later on, even though my HLSL settings are not there yet.

    - GroovyMame changes the way emulators work, and the specific order in which they do run the steps of emulating, making it accurate, but built in a more logical way, perfomance wise.

    Bottom line, so far, both emulators seem to get the same results regarding frames of delay.
    But, GroovyMame devs seem very responsive and striving to get to the point where you won't have any lag coming from the emulator (while flagging the outside causes and trying to find solutions) while remaining accurate. Great project. But seems like a pain in the ass to use for now. so we can hope that at some point they'll get a guy and more things to make it user friendly.

    So right now I'm using Shmupmame, and even though I do have a laggy TV, it's getting damn close to the arcade, IMAHO.

    Hope some of this makes sense.

    I'll add links to threads worth reading if you want to check the detail of what I tried to explain there, later on.

    [EDIT] :::

    - Here a link to Papasi's thread here on SRK where he made all the same lag tests and more, specifically on ST using his PCB and Shmupmame as comparison.

    - Somwhere along the thread, Jdubs came in and linked to a thread where he started to talk to GroovyMame's devs about input lag. They've been very responsive and tested a lot of thing and actually made some great discoveries that should largely improve their next release. Very long but interesting read. btw, I'd have to check again to confirm before spreading bad info, Seems like in this thread, Dark Gaiden has tested the influence of bilinear filtering and claims it adds some lag. But I can't say if he tested it or if it's a hunch, I'll message him to see and edit later. Hence me posting screenshots below with and without this option turned on as it's fairly common to use it to make the game lookable on a LCD (not needed at all with HLSL)

    [EDIT 2] :::

    - here is also a link to the official ShmupMame page, it is pretty short to read and understand the means the dev uses to get it going.
    Post edited by alien nose job. on
  • ilitiritilitirit Joined: Posts: 6,388
    Can you also please post which input methods (and the brands/manufacturers) you tested? In my experience the MadCatz TE PCBs are slightly laggier than some other brands. I don't know if they're universally laggier across all platforms or just on PS3 though.

    It's pointless to argue over 1f of monitor/emulator input lag if your controller is just as laggy.
  • alien nose job.alien nose job. Joined: Posts: 88
    Sure, I used both an old crappy Hori Ex2, and a HRAP VX SA for Xbox360.

    Using USB sticks on emulator seems to add 1 frame of lag, from what I read it should use keyboard material or some sort of bridge to get rid of it, but this is not my point. But it seems that the guy that works on GroovyMame is aware of the issue, and will ask official mamedevs to change it in future iterations to make implement RAW input for sticks too, not just keyboard.

    I just meant to say Shmupmame (and most likely GroovyMame) are an improvement on FBA and old Mame.

    Don't get me wrong, I don't mean to fuel a flame war of what's good what's not, like PC sucks or rocks.
    I just think the more access we have to better versions of the game, the better.
    And it seems like something cool aside console, which is good in other ways but also sucks.

    As promised, here are screenshot comparisons of different graphic options on Schmupmame (or any emu right now I think) to display HLSL:

    TESTHLSLCUTbigger01_zpse04cd48c.pngTESTHLSLCUTbigger02_zpsa5d1983d.png

    Sorry for the big image, spoiler tags don't seem to work.

    Please bear in mind I edited the HLSL settings more or less as I could, there's slightly too much blue and red, I have to get back to the arcade to check again.
    Still, it does look a lot more like arcade, and way better than the regular scnalines filter, which is great if you're stuck with a LCD.

    By the way, I mentionned Bilinear filtering as it seems that it also adds lag. Yet I have to check again to confirm.
  • ilitiritilitirit Joined: Posts: 6,388
    I was actually referring to the tests performed by the OP, but thanks anyway.
  • isotopezisotopez Joined: Posts: 125
    ilitirit wrote: »
    Can you also please post which input methods (and the brands/manufacturers) you tested? In my experience the MadCatz TE PCBs are slightly laggier than some other brands. I don't know if they're universally laggier across all platforms or just on PS3 though.

    It's pointless to argue over 1f of monitor/emulator input lag if your controller is just as laggy.

    Not sure what you mean by "input method," but I listed almost everything I did in the original post. 360 was on hori pcb. Ps3 and emulator on mc Cthulhu.
  • ilitiritilitirit Joined: Posts: 6,388
    Weird. For some reason my browser caches page 2 as page 1, so I only see the first post from page 2, not the original that listed the controllers. Anyway it's sorted out now. I had to clear my browser cache and log in again.
  • Lance3rdLance3rd Alaska 3rd Strike Joined: Posts: 986
    bk2099 wrote: »
    isotopez wrote: »
    I've finished rerecording and recounting footage for the 360 and dreamcast. I decided to only count the frames at 120fps and then divide those numbers by 2, which gives the lag at 60fps with half a frame of precision.

    The videos for the consoles aren't consistent with cps3 because they refresh at a different rate, so the console averages are likely skewed a little compared to cps3.

    These numbers do not mean that 360 lags more than the ps3 or even cps3. Both consoles' averages fall within the same range of error.

    3S Lag Results:
    CPS3 - 3.6
    PS2 - 4.7
    PS3 (filters off) - 3.8
    PS3 (filters on) - 3.8
    360 (filters off) - 4.1
    360 (training mode) - 3.9
    Dreamcast - 6.1

    *All these averages should be +/- 0.5 frames of the actual lag.




    I was wondering if you could retest the Dreamcast version when you get some time. I would do it myself but I don't have the recording gear.

    The reason is because the CPS3 hardware uses a Hitachi SH-2 as its CPU and the Dreamcast uses the Hitachi SH-4, which is in the same processor core family. This makes the Dreamcast the closest to the CPS3 in terms of hardware.

    Also if you could perform the tests:
    -in 240p mode [Hold Start+L (or Start+Z on the Arcade Stick) before the Capcom logo appears to get the 320x240 mode.]
    -using a "Madcatz Dream Pad" or any other 3rd party controller since the stock Dreamcast controllers are known to lag.
    -using the Street.Fighter.III.Third.Strike.Arcade.Edition-RDC ISO. Apparently it's arcade perfect with unblockables.
    -bonus if you can wire it up to an arcade monitor in 15khz RGB mode.

    I'm not sure what isotopez' thoughts on this are - from my perspective, I think we could do it but it would be really low priority. it's taken a good while to actually set up the speed tests (plus we both work so it hasn't always been our top priority lol) and we plan on actually doing those pretty soon. we've thought about going back and re-testing for input lag with isotopez' more accurate method after we're done with the speed stuff, and we still might do that.

    IMO Dreamcast and PS2 are kinda bottom of the importance list. there's more to be gained from testing the big platforms that a lot of people use - CPS3, OE, and emulator. not saying we won't do it, just won't be at the top of the do list.
  • isotopezisotopez Joined: Posts: 125
    I doubt we'll do anymore tests with the dreamcast. There was 1 thing I still need to find out for esn, but other than that, it seems bad enough that no one should be playing 3s on it. I don't have access to a ps2 anymore, so we probably won't run speed tests on it or retest the lag with a stock controller either.
  • TiredOceanTiredOcean Low-tier theory fighter Joined: Posts: 134
    Interesting find isotopez. Although I can't say I understand all of this, would the main difference in speed between the console and CPS3 versions be due to the difference in refresh rates, or would you put it down to how 3rd Strike is emulated in OE?
    Some simple maths shows that, if we assume the refresh rates are the only difference, CPS3 3S runs at ([59.583*100]/60 =) 99.305 % the speed of OE. If only there was a way to hack the game to slow it down by this amount...
    I've tried running shmupmame at 0.99305 speed but it still feels too fast, but I haven't tried any in-depth tests.
  • Lance3rdLance3rd Alaska 3rd Strike Joined: Posts: 986
    I think it means the perceived difference in gameplay speed is just down to refresh rate. I expected going in that we'd find a smoking gun like "you can cancel Chun's low forward into super on CPS3 much later than on OE" since it definitely feels that way, but it didn't seem so. you can tell from how the tight links dropped on OE that there's definitely a timing difference, and yeah for anything that requires tight timing several times in a row it'll eventually drop.

    so I think it's fair to say what people have always said - OE is not the same as CPS3 but it's reasonably close. if you play characters that require tight links you'll probably notice the difference. if you don't, you'll probably have a slight bit more time to react to things on CPS3 but it's not huge.

    we haven't tested audio latency at all, and that could be a factor too. if OE had a longer audio buffer you could be hearing your hit sounds later and would have less time to confirm if you confirm off sound. it's definitely possible. any stage or move that causes slowdown on CPS3 will make a difference as well. and of course if you're playing on a laggy stick or monitor or playing online, things will be really different (goes without saying).

    also regardless of our results, OE's sound effects still suck. there's no cure for Oro snoring and trashcan shoryus.
  • isotopezisotopez Joined: Posts: 125
    edited June 2014
    The games do run at different speeds, so if you're a robot with a super accurate ability to push buttons at specific times, you might notice the difference in speed if you try doing long combos like elbow cannons, tth on the twins, or some very timing specific genei jin combos. Whatever combo you're doing though has to last long enough for your inputs to desync with the game speed, like tebbo was saying. That of course only matters if you're used to playing on 1 platform and you switch to the other. Most people probably can't do x6 elbow cannons on urien consistently without double tapping buttons. If they can though, they have visual and audio feedback that they probably depend on more so than an internal clock in their head. I can't say that is true about everyone, but for most people it is.

    I think there may also be a perceived difference in timing just based on how easy it is to get your input to come out on the frame you intended it to come out on between cps3 and oe. Perhaps USB on 360 is prone to occasional lag, which would change the timing of your inputs. This would only become apparent if you press your button on the very edge of cancel or link windows i.e if the move comes out 1 frame late you drop your combo.
    Post edited by isotopez on
  • yuukiyuuki Joined: Posts: 782
    "The games do run at different speeds"
    "'May be a a perceived difference in timing"

    If the game runs at a different speed then it's not a perceived difference in timing but actually a difference in timing
    かかってきな。
  • isotopezisotopez Joined: Posts: 125
    I added an "also" to my post (May also be a perceived...). I see them as 2 different reasons people will say there's a timing difference. The actual timing difference is very minute whereas a perceived difference isn't really a timing difference, but it'll feel like one and it would probably be more noticeable.
  • TebboTebbo Play. Joined: Posts: 5,702
    that plus any input lag differences make it easy to notice/feel.
    it's possible to sense something which measurements might tell us is essentially negligible. humans are remarkably acute.
    Play more.
  • TiredOceanTiredOcean Low-tier theory fighter Joined: Posts: 134
    Well...... A lot of audiophiles claim they can "sense" differences between music files/sound systems that they really can't (i.e. the difference between a 320kbps MP3 vs. lossless audio), which lead them to spending thousands on equipment that don't really make a difference.
    But that's perceiving differences in sound quality, while we're looking for differences in speed. Humans might be a bit better at that.
  • TebboTebbo Play. Joined: Posts: 5,702
    so you're saying without knowing already that we couldn't tell the difference?

    we might exaggerate sure. but the game feels different. 360 OE on a crt is pretty good though.
    not everyone might notice, but some probably do 100% of the time. there is 3S and then there are lesser versions of 3S.
    even if its small, differences exist and they are noticeable. they do add up. it's immediately apparent in my experience. super links are an easy example.

    so like isotopez is saying. it might be pretty small, but the actual effect of say 1 frame of input lag more, can be pretty enormous (perceived). especially if the gameplay is running faster and the inputs are running slower (like the ports). so in the end you just have that much less time to react to something. teching throws is easier on cps3, period.
    Play more.
  • DanderDander BANNED Joined: Posts: 7,227
    sharp in contrast deez nuts
    8tROOXi.png
  • Lance3rdLance3rd Alaska 3rd Strike Joined: Posts: 986
    http://forums.shoryuken.com/discussion/191312/how-much-does-your-arcade-stick-lag-arcade-stick-input-lag-testing-results-thread

    just wanted to post this here. it's a rundown of various arcade sticks and their innate input lag. as you can see every arcade stick varies. just one more thing that could add up to a different experience. for instance when OE first came out there was a general feeling that OE lagged more on PS3 than 360, but this could just as easily be arcade stick as anything else. even a stick that is modded for several different consoles can have different delay on each platform! or a stick with no input delay on console might have poorly written drivers for PC and have delay there.

    every individual component in the chain adds up to the overall experience.
Sign In or Register to comment.