That's not that big of a difference.
My brother is 29 (30 on Dec.1) and he started dating his chick when she was 19 (20yrs old now). That's a 10 year difference. I say go for it. 24 is still young. I'm 22 and feel "old" sometimes, but I have to keep reminding myself that I ain't. Just be a sick man and go for it. Shit my grandpa and my grandma were 13 years apart. lol