Is that rude? Maybe. But, I mean, I'd never hold anything I overhear against the people who are saying it. I don't know them, so I'm not judging anything I hear...I just might use what they are saying (or the gist of it) as part of the dialog in my book.
Or, if the conversation they're having is juicy enough, it might even be a whole new idea for a project.
It's not just at restaurants, but it could be in public lady's rooms, fashion stores, waiting in line at the cashier or the bank, or anywhere people will speak with each other, or on their phones (although I'm only getting half the conversation, and I have to use my imagination to fill in the holes).
Have you ever come up with some great ideas just by listening to what others are saying?