Mass Confusion

The first time I spent any time with him, I came away from it not liking him in the least. I thought he was crass and annoying in fact. And even now I sometimes still do. But somewhere along the way, my feelings changed entirely. I somehow became attracted to him. No, not "somehow", I know exactly how. He does that witty banter competitive conversation thing, and he's good at it. A quick mind is always attractive to me. And his eyes, my goodness does he ever have amazing eyes. At least he isn't really hot, or I might have been in trouble fast. Unless that made him actually seem attainable. Maybe if he had been really hot, I would have been able to just have a fun little crush and move on, knowing that nothing would ever come of it. No, instead, we have spent time together and there was a time when he flirted with me and there was a palpable tension between us, so much so that someone asked him about it. He was embarrassed, and who wouldn't be. Thing was, I had a boyfriend at the time, which complicated matters. But that day, he asked me, "Am I being too obvious?" I panicked like a deer in headlights and played it off like I didn't know what he was talking about, but I did leave him a note that said, "You aren't obvious, but you aren't subtle either." He decided that I was bluffing, and then we had a silly little try-to-get-the-other-person-to-say-it-first-thing. That was dropped after about a week, and he never did tell me exactly what he had been asking about.
We have only hung out like just the two of us on two occasions, both at my house since it's on the way to his. The first time it was normal, but the second time there was a lot of talking about things by metaphor and innuendo. When he went to leave, I went outside with him and we ended up talking for another hour in the cold. And I didn't mind one bit.
Things are much different now. The same person who asked him what was going on between us at some point told him that if he were to ask me out, I wouldn't say no. I didn't know about that until after, and I really wish that she hadn't. Shortly after that, I broke up with my boyfriend. It really had nothing to do with any of this, I had needed to for a while and I finally got the courage to do it. Among my friends, the guy that I had the crush on turned out to be one of the most helpful when I was trying to work things out. Granted, he did take a few jokes too far too soon, but for the most part he was great.
The thing that confuses me is that now that I am essentially available, he has almost completely stopped flirting with me, or really expressing any interest whatsoever. I still see a little interest in his eyes (good lord, his eyes) but that could just be because I make him laugh in general. He will say things occasionally that make me wonder, both ways. Perhaps he is just waiting until enough time has passed. He is going on a two week trip around the country soon. That might have something to do with it. On the other hand, every time that I invite him over or to an event, he has said no.
Maybe I am just a fool, looking for something that isn't there and seeing signs of it in my imagination.