The context is I’m talking to my brother and he is doing something else and perhaps doesn’t hear what I say. I ask, “Why don’t you reply/respond to me?” Are both words correct? How would native speakers put it? Thanks.

Why aren't you answering me?

In the context, there are a couple options. "Did you hear me?" Is a good choice. If you were already speaking and he suddenly stopped responding, most native speakers would simply say "what?", or possibly "what's wrong?" , which is asking why they have suddenly stopped responding. Or they might say "hey, are you (still) listening/did you hear what I said?". There are some more playful phrases you can use if you and your brother are close, like "do you need to get your ears/hearing checked?", but again these are only to be used with close friends or family who don't mind a little playful teasing. The examples you gave are grammatically correct, although you would probably say "why didn't you respond/reply to me?", using the past tense. However they probably wouldn't be used as much by native speakers. More commonly you would hear "why aren't you answering me?", as "respond/reply" is used more when referring to things like text and email. "He replied/responded to me" would usually mean "he texted/emailed me back", They are both also used for speaking, but it can sound a little formal or mechanical to use them in that context when speaking with another person. Edit: also you need to make the distinction between "aren't" and "didn't", since the first is used for things happening right now, and the second is usually referring to an event that happened in the past.

I would more likely use respond, but if we are texting, I might say reply (but very unlikely in speech for me). But someone also mentioned "Why aren't you answering me?" and that feels more natural.

'Why won't you respond (to me)!' in the imperative mood

EIther reply or respond is fine, the "to me" is unnecessary because it's implied by the context.

Both of those are really too formal for a conversation with your brother. “Why aren’t you answering (me)?” is the absolute best way to frame this, I’d say. “Reply” and “respond” aren’t words you’d typically hear most English speakers using in normal causal conversation regularly. That’s not to say they would never be used, but “answer” is a more natural term in that context. Informally, “reply” and “respond” would be more so used in a text/messaging/online context as opposed to in person casual discussions. You’d also likely find them in formal situations, especially structured ones. You may “respond to an important email” or “reply to a difficult question”, but those aren’t really terms you’d use for familial communication.

“Don’t” is incorrect here. It sounds like you are making a suggestion or giving advice that they should start responding. You can use “won’t” in its place, and that would make sense. Or, “Why aren’t you responding?” But “respond” and “reply” are very formal. You would say “answer” to a brother (without the “to me”). 

A native speaker would be much more likely to say, "Did you hear me?"

Both are perfectly fine but sound too formal for a brother. In spoken American English, we'd be most likely to say "why don't you answer me?" or "why aren't you answering me"

If you think he didn't hear you, just ask "did you hear me?" Using "Why don't you respond?" is used when someone does hear you, but is not answering the question as a way of trying to avoid or control an argument. edit: that is, in my experience; others may have other suggestions

I would likely say “Did you hear me?” “Why aren’t you saying anything?” “Why haven’t you answered?” or just their name, loudly, to get their attention “**Pat**!”