I wasn’t planning on buying the fourth generation Apple TV right away. But then Guitar Hero Live (Free) happened, and I just couldn’t wait. I mean, the game is great on the iPad and iPhone, but obviously it was always meant for the living room. Imagine my disappointment, then, when I finally got Apple’s new streaming box and Guitar Hero Live wasn’t even in the App Store yet. Luckily, though, developer Harmonix (who kicked off the whole plastic instrument trend back in 2005) did have a rhythm game on the store right on launch day. It’s called Beat Sports, and while it’s no Guitar Hero or Rock Band, it’s still a pretty fun little slice of music gaming.

In Beat Sports you play as a “slugger” who is apparently a child living in outer space with little more than a baseball bat. Across from you is a variety of aliens that throw, spit, or eat balls, and your goal is to whack these balls back into their adorably hideous mouths. The twist (if any of that even needs a twist) is that you must time your hits to the beat of whatever cheerful tune is playing. It’s a super simple mechanic, since unlike other rhythm games, here you (usually) only have to worry about a single hit instead of needing to coordinate several different buttons or pads or whatever.

Here’s another twist: Since this is an Apple TV game, and the new remote has motion sensing capabilities, in order to “hit” the balls you need to physically snap the remote into the air to mimick a good thwack. On paper, that sounds an awful lot like Wii Sports (especially since the first game you learn in Beat Sports, called “Net Ball”, is basically tennis), but in practice, it’s pretty different. Mainly because in the former you had a surprising amount of control over how hard you hit the ball and where it went, whereas in the latter you have no control at all except for when to wack. That’s not necessarily a bad thing, though, as long as you don’t go in expecting more Sports than Beats.

The other two game types, “Whacky Bat” and “Hangry Holes”, are only slightly more complicated. Whacky Bat sees you hitting the ball to the beat as you swipe the top half of the remote to slide your character back and forth between lanes. Hangry Holes does away with the motion controls altogether and relies on swiping to hit the ball, and sometimes clicking as well. Each of the three game types is harder than the last, and unfortunately that’s where the problems start to crop up.

I noticed it pretty much right away when I first dove into Net Ball: The dreaded rhythm-killer known as “delay”. I’m not sure if it’s the game, the Apple TV’s remote, or just the limits inherent in swinging your hand around, but there’s a noticeable split-second-or-so between your action and the action on screen. It seems like the game knows this and tries to compensate, but it doesn’t always work. It’s not a major problem in Net Ball since all you’re doing is jerking the remote to the beat, but when you start adding swipes and clicks into the mix things get really messy. Especially for Whacky Bat and Hangry Holes, I had to force myself into the habit of doing everything slightly before the beat in order to hit the balls in time, which gets really complicated later in the game when you’re trying to listen to the beat on the TV while simultaneously shifting that beat ahead an eighth or sixteenth note in your mind as you prepare your shots. To make matters even worse, when the difficulty (and tempo) ramps up, the game seems to have more and more trouble discerning your waggles and swipes as they get more rapid and close together. (Interestingly, you can usually “click" the remote instead of swinging it, and while it’s not quite as fun, it’s definitely more reliable.)

Luckily, it takes a while before the single player mode starts to get that frustrating. Plus, if you do start tearing your hair out, you can always jump into the game’s multiplayer mode with a friend. It’s called Buddy Ball, and it can be played with up to four people. I tried it out with three people total, and it was pretty neat once we discovered all the ins and outs. Basically, you each take turns hitting the ball across the arena to some aliens that are either yellow, purple, or orange. They will spit the ball back to the next person in line quickly or slowly (based on the color), and it’s good fun trying to trip up your neighbors. Unfortunately, the same delay problems crop up here, compounded by the fact that the rest of your friends have to use an app on their phones (Beat Sports Remote (Free)) connected through wifi. Again, it can all be compensated for with a little practice, but it does undermine the “rhythm” aspects of the game.

Beat Sports by Harmonix is a game that should be crazy amounts of fun. Think about it: it was only 7 or 8 years ago that Wii Sports and Guitar Hero were both the go-to party games when you had friends and family over. Imagine combining them! Unfortunately, shaking a remote around is just not precise enough to work reliably in a rhythm game. I mean, there’s a reason most successful ones are based on buttons, taps, and stomps rather than arm flailing. Still, Beat Sports has a ton of character and delightfully catchy music, and still manages to be quite fun despite the slight delay issues. Hopefully some of that can be minimized with future updates, but until then, it’s still a great showcase for the unique experiences Apple’s latest platform can offer.