A word of warning here: Make sure you don't do this too abruptly or shift into negative body language, or he'll think you're suddenly not interested. People mirror bad body language, as well, and if you suddenly sit back, cross your arms and look down your nose at him, chances are he'll think the game is over and do the same! Instead, try something like leaning over and swirling a straw in your drink, placing a hand, palm down, on top of the table closer to his side than yours, etc—something he could easily mirror without looking silly.
If he smiles back and continues doing all he's been doing, rest assured, you've hit a home run. If, however, he makes an excuse to leave at this point, that means he was just flirting for fun and not for an end result—and you just upped the odds to stakes that were too serious. (Never mind, better to find out earlier in the game than later!) If all else fails and he still doesn't ask for your phone number when you're about to part, take a deep breath and ask him for his! It could simply be a case of him being shy or thinking you're way out of his league.