When you speak, you need to ensure that the attention of the audience is focused on you. One way to do that is to be an engaging and entertaining speaker (which you are already, of course), but you also need to think about removing any distractions. Anything which competes for the thoughts or feelings of the audience will reduce the impact of your words. Here are some common distractions, and how to tackle them:
1) Noise. Any external noises, such as speakers in an adjoining room, noise from the street, or (for an after-dinner speech) tables being cleared, can cause a huge distraction. It’s not always possible to eliminate sources of noise, but you should always check for noise (and the likelihood of it) when you arrive early for your sound check. Windows can be closed and the meeting organiser can ensure that staff are not serving or clearing tables during your speech. If you are using a sound system, check whether the volume can be increased to overcome external noise (not to rock concert levels, obviously).
2) Timing. Never, ever run over your allotted time. That will make the audience very uneasy, and many will lose interest as they yearn for their break. Always be ready a few minutes ahead of time in case you are called early. Watch your time carefully throughout your speech.
3) Hecklers. Ideally, this will never happen to you, but if it does, preparation is the answer. In a comedy club, you will be able to deliver a withering, pre-prepared insult. In a conference speech, it is better to ignore a heckler, and allow the organisers to deal with them. You may need to pause, but don’t be diverted from your message.
4) Environmental factors. The room could be too hot, too cold, or the lighting may be so poor that you can’t be seen properly. Again, it’s about planning. You need to do your best well ahead of time to minimise any problems, and to make sure that there is someone on hand to sort things out while you carry on calmly.