HDMI out and gaming

Discussion in 'General Game Discussion and Questions' started by killercow, Dec 18, 2012.

  1. I am wondering if ios 7.1 had any effect on the iphone 5 experience. I am not sure if anything changed on HDMI recently, but since ios 7.1 had many improvements and since HDMI lightning stresses the cpu with the airplay protocol conversion to sub HDMI quality, I think that maybe it.

    It is good news to hear that even pinball games are working well on your HDMI setup, it is true that before that almost all HDMI games played pretty well on HDMI, but of course some really fast paced action games like pinball games were slightly harder to play because of the slight lag inherent in HDMI.

    The most scientific informarion were made by Menneisys, and also that one Panic article about Lightning HDMI, those documents can be found on my Toucharcade TOC (Table of Contents) here:

    http://forums.toucharcade.com/showthread.php?t=173262

    We need more HDMI reports from ipad air or ipad mini retina users asap here!!!
     
  2. Adams Immersive

    Adams Immersive Well-Known Member
    Patreon Bronze

    Dec 5, 2008
    1,718
    5
    38
    Freelance interactive design and programming
    Ohio
    #462 Adams Immersive, May 29, 2014
    Last edited: May 29, 2014
    Interesting.

    Maybe there's evidence of the firmware version having changed in the HDMI adapter, as a little-known part of an OS update?

    I agree: a return policy is safest! But even at its laggiest (still only a fraction of a second) Lightning HDMI was alreadyl fun for many kinds of games.

    I don think it's Apple-like: Lightning is a whole new kind of connector, which adapts the pins to any purpose on the fly, and this is VERY future-proof, for video (and non-video? medical?) connections that nobody has yet even thought of. Great tech, and I think it's a good plan.

    One side effect is a slight lag that only matters for very rare gaming situatios (because it doesn't matter for all games, and most iOS gamers never output to HDMI at all; it's a neat thing and supported, but niche). Wired-video twitch gaming may be a growing niche for now, but not enough for Apple to change the whole Lightning plan over. Their plans are too big and too long-term, and that one HDMI-lag factor is just one among so many. And a factor that will improve over time anyway--even if that means a different cable one day.
     
  3. I think the ipad cpu has an extra load now compared to the old nearly flawless 30 pin HDMI. Since lightning HDMI is not true HDMI but a junky downscaled resolution airplay based protocol, I think there is a load on the cpu which is apparent in games with framerate issues. The old 30 pin did not have framerate issues, only a slight lag.

    Lightning HDMI has even worse lag, but also it has framerate issues.

    Now, ios 7.1 or having a newer ipad with 64 bit ability should make the lightning conversion faster I would think. So while the lag may not be decreased that much, the framerates should in theory. Of course, I have not seen any scientific proof of any of my speculation, but many were saying in the past that lightning HDMI should improve with better cpus and gpus. And there is a chance that ios updates like ios 7.1 may improve lightning HDMI too.

    About HDMI, buy a 25 foot HDMI from Amazon, they are dirt cheap and work great.

    Apple and most electronic stores have flexible return policies for HDMI cables and the Apple adapter. You can buy them, and try them out and either keep them if you like lightning HDMI, or return it if the lag or framerates are not acceptable.

    I would definitely give it a try. In fact, I would encourage you to try that. Make sure you buy the latest Apple HDMI adapter and not a knock off since the knock offs have a huge problem. But the generic 25 foot HDMI cables work great because there is no circuitry there.

    If you get lightning HDMI for you ipad mini retina, please let us know how it is!
     
  4. By Selachii, dev of Phoenix HD

    http://forums.toucharcade.com/showpost.php?p=3242959&postcount=1509

     
  5. kmacleod

    kmacleod Well-Known Member
    Patreon Silver

    Jul 1, 2009
    1,865
    1
    36
    Artist / Writer / Designer
    California
    For those curious, I have not noticed any significant change between iOS 7 and iOS 7.1 over HDMI.

    Graphics quality is generally good, framerate is consistently smooth, but lag is still noticeable.

    Basically, most games are fine, but twitch-response games border on unplayable.

    The current beta of iOS 8 doesn't change anything on that front, though it does allow you to use a regular Lightning cable to output video to your Mac, the same way the HDMI cable outputs to the TV. The quality appears identical.
     
  6. Jorlen

    Jorlen Well-Known Member

    Jan 7, 2009
    1,094
    2
    38
    IT
    Canada
    Thanks for those impressions.

    So it looks like there are no planned improvements for the lag issue. Likely, what we see now is the best we'll get, when you consider that the video out is being processed by a chip in the cable itself.

    I took another look at a few youtube videos of people doing this. It's very obvious because you can see the ipad screen & the TV at the same time, and the lag is almost a full second behind. I'd say maybe half a second, which is not something I'd be able to adapt to for twitch games (which is what I'm after on a big screen mostly).

    I can understand that not a large % of people with IPADs will want to output to their TV for games of this kind. But I was definitely looking forward to it and would have purchased an MFi controller for sure. Now I think I'll just keep using the ipad. Lag isn't something I can tolerate.
     
  7. Adams Immersive

    Adams Immersive Well-Known Member
    Patreon Bronze

    Dec 5, 2008
    1,718
    5
    38
    Freelance interactive design and programming
    Ohio
    There is DEFINITELY some variation, whether from game to game, iPad to iPad, HDMI adapter to HDMI adapter, or TV to TV.

    I have tested again, and I am not crazy: my HDMI lag is MUCH more slight than when I first tested last year, and is suitable for all but tht twitchiest games. It's suitable for Real Vampire Pinball. When I first tested last year, the lag was super noticeable and beyond what I could get used to for most games. (Likely still true with AirPlay, but I don't have Apple TV.) But this I can used to for almost any game--and in fact, can't notice at all with most games. I notice the lag most clearly when simply swiping the home screen back and forth. Swipe quickly and you'll see a slight delay.

    The lag is real, but FAR less than a half second. (With my iPad Air, iOS 7.1.1 and one of the earliest HDMI Lightning adapters.) It's about 1/10 of a second.

    I ballparked this as follows:

    - I used SoundGrid (or use any grid-based music app with a sweep that you can set the BPM).

    - I held the iPad up half-covering my projector screen so the grids were aligned in my eyes, and watched the sweep.

    - I adjusted the BPM to try to make the sweep bar on the projector lag exactly one unit behind the one on the iPad. (In other words, the two bars move visually exactly at the same moment--but one beat delayed. If the Projector bar is sometimes briefly 2 units behind, slow the BPM. If the projector bar sometimes catches up brieflt, speed up the BPM.)

    Result: the closest I could get was the max BMP of 500. And it's very close, but the projetor bar catches up just BARELY for an instant on each beat. So 500 BMP (or 8 1/3 beats per second) is just a bit slower than the HDMI lag. That's where I get my estimate of 1/10 of a second. I'll try to test via on-camera measurement with a numeric timer running later.

    But yet some people DO have much more serious lag. They're not crazy either!

    What is the difference? I'm curious.

    One factor could be that some TVs actually add lag (unless put in a "game mode" or otherwise disabling various processing/enhancements that makes TVs look nice in the showroom).

    That doesn't explain the difference for me from last year: I'm still using the same projector (Qumi Q2). So has there been a firmware update for the adapter, or a relevant software improvement in iOS? I have to wonder.

    It could also be that some games add lag, straining the iPad CPU/GPU more. Or, maybe even some kinds of imagery strain the processor in the HDMI adapter more (sounds doubtful to me: I'd expect quality, not timing, to slip in that case).

    Some theories I tested that are NOT the explanation:

    - Screen Zoom enabled (but zoomed out so you can't tell)? Sometimes I do that and forget. But I tested with different zoom levels and see no added lag that I can detect.

    - Apps that mirror could have worse lag from ones that output a special image just for video-out. But Real Vampire Pinball is mirrored.

    - Rebooting to clear memory? I didn't do that, so that wasn't the solution.

    Note: complicating this is the issue of framerate: a slow framerate would make the problem worse, adding additional latency. But all my tests were things that run really smooth, taking that mostly out of the equation.

    P.S. This may be my final test... I think my HDMI adapter died! The picture and sound started to crackle with the slightest movement (like a weak connection) and now it doesn't work at all. The cable tests out fine. :( Hope I bought it on a warranty-doubling credit card...
     
  8. kmacleod

    kmacleod Well-Known Member
    Patreon Silver

    Jul 1, 2009
    1,865
    1
    36
    Artist / Writer / Designer
    California
    That matches up with what I've noticed, though you've gone much further than me in documenting specifics.

    Lag, for me, is consistently about 1/10 to 1/5 of a second. Nothing seems to change it, as you've found. It is independent of memory, what's happening on screen, reboot status, any of that. It seems in-built in whatever processing is happening to obtain the H264 signal.

    The use of H264 could easily explain this. Much of the quality of H264 comes from its use of predictive frames - basically, the video compresses itself much better by looking at the the next 3-5 frames of action. Apple could be intentionally delaying the video output by several frames to improve video quality.

    Video output over Lightning has interesting constraints. There is a hard limit in bitrate possible, which is the bitrate of the USB 2 data connection used by Lightning. The iPhone needs to compress the video to look as good as possible, but still stay under that limit.

    Without predictive frames, they may not have been able to obtain adequate visual quality from the H264 codec.

    Interestingly, modern WIFI can have a higher bitrate than USB 2. If Apple were to tweak the Airplay protocol a bit and include stronger antennas in future iPhones and iPads, they might find that they can get superior quality from over-the-air streaming than is possible over Lightning.
     
  9. Adams Immersive

    Adams Immersive Well-Known Member
    Patreon Bronze

    Dec 5, 2008
    1,718
    5
    38
    Freelance interactive design and programming
    Ohio
    #469 Adams Immersive, Jun 11, 2014
    Last edited: Jun 11, 2014
    The decoding in the connector's chip is certainly a source of delay (probably THE source of delay) but I don't think Lightning is limited to USB speeds--it doesn't use USB, except of course when you use a Lightning-to-USB adapter (the charging cord). Lightning is adaptable to future and unkown standards (the reason for its design), and I don't think we know its limits. (One early teardown suggested that it may be USB 3 capable, and I don't think we know that for sure either way yet.)

    Two ways the delay could be reduced further--and both seem inevitable in the long run, as newer parts eventually become cheaper than old ones:

    1. The CPU in the adapter gets faster

    2. The CPU in the device (preparing the data to send to the adapter) gets faster

    Software/firmware improvements are also possible in theory (even the use of different codecs) but less certain to happen.
     
  10. Selachii

    Selachii Well-Known Member
    Patreon Silver

    Nov 13, 2009
    659
    6
    18
    Game Designer
    Netherlands
    #470 Selachii, Jul 10, 2014
    Last edited: Jul 14, 2014
    We have just released our "Big Screen" update for Phoenix HD and have really experimented with all output methods/resolutions etc. There is a lot to gain for developers, but it does take some time and effort to get everything right. There's also some work on Apple's plate here, but we get the impression they are aware of this :)

    The performance for AirPlay, the 30-pin HDMI adapter, the Lightning HDMI adapter and the AV adapter were all individually tested and tweaked. We also made sure MFI and iCade support worked seamlessly to create the perfect 'console' experience.

    To be a bit more specific:

    • AirPlay runs at 30fps on 720p. It works on iPhone 4S, iPad 2 and everything newer than those. The performance is not optimal on those older devices, we recommend it on an iPhone 5, iPad 4 or newer. The quality of the WiFi network is also quite a big influence.
    • The 30-pin HDMI adapter runs at 60fps on 1080p and looks amazing :) This works on iPhone 4s, iPad 2 and iPad 3 as you would expect.
    • The Lightning HDMI adapter also runs at 60fps on 1080p on all devices you can plug it into, but as you all know doesn't look as good as the 30-pin counterpart. Still though, this looks rather amazing on a big screen.
    • The AV adapter works the same as the HDMI adapter but also supports 1024x768 at 60fps in case you hook it up to a 4:3 display.

    We tested and tweaked every adapter with nearly every device, but we are always keen on input. So if you have any feedback, I'd love to hear it <3
     
  11. Adams Immersive

    Adams Immersive Well-Known Member
    Patreon Bronze

    Dec 5, 2008
    1,718
    5
    38
    Freelance interactive design and programming
    Ohio
    That sounds great! Thanks, Selachii.
     
  12. kmacleod

    kmacleod Well-Known Member
    Patreon Silver

    Jul 1, 2009
    1,865
    1
    36
    Artist / Writer / Designer
    California
    Congrats on the Phoenix HD update! I played it before over HDMI using an MFi controller, and it worked pretty well. Glad to see a focus on making it work even better.
     
  13. darkarn

    darkarn New Member

    May 6, 2017
    4
    0
    0
    #473 darkarn, Jun 5, 2017
    Last edited: Jun 5, 2017
    I noticed some strange lags (extremely brief and random moments of visual lag (about 1 second), no lag for audio) on the HDMI output for devices updated to iOS 10.x when I use the Lightning to HDMI adapter. They look like dropped frames too.

    Anyone else here having similar issues?

    (Three years old thread I know. I am just unsure where else can I ask the issue)
     

Share This Page