How can I convince my mum that paying people for prayers is not right, she is just being ripped off? She does not want my brother to go back to prison so she believes paying people to pray for my brother -- and I mean a lot of money -- will prevent him from going back. When I tell her it's wrong, she refuses to listen and says I don't understand, then avoids conversation.
Generally most people try to tell someone else what is right or wrong. But because they don't take the conversation back to the Scriptures, it sounds like all you are offering is your own opinion, even if it is based solidly on God's teaching. "If anyone speaks, let him speak as the oracles of God. If anyone ministers, let him do it as with the ability which God supplies, that in all things God may be glorified through Jesus Christ, to whom belong the glory and the dominion forever and ever. Amen" (I Peter 4:11).
Your mother has gotten caught up by con men who are preying on her fears. "... having a form of godliness but denying its power. And from such people turn away! For of this sort are those who creep into households and make captives of gullible women loaded down with sins, led away by various lusts, always learning and never able to come to the knowledge of the truth" (I Timothy 1:5-7).
It is true that we can ask Christians to pray for us. "Confess your trespasses to one another, and pray for one another, that you may be healed. The effective, fervent prayer of a righteous man avails much" (James 5:16). But whether your brother ends up in jail is based on your brother's choices in his life. "The soul who sins shall die. The son shall not bear the guilt of the father, nor the father bear the guilt of the son. The righteousness of the righteous shall be upon himself, and the wickedness of the wicked shall be upon himself" (Ezekiel 18:20). Fortunately we are not in a period of history in most parts of the world where people are put into jail for doing righteous deeds. If your brother is doing what is right, there is no need to fear the police. "For rulers are not a terror to good works, but to evil. Do you want to be unafraid of the authority? Do what is good, and you will have praise from the same. For he is God's minister to you for good. But if you do evil, be afraid; for he does not bear the sword in vain; for he is God's minister, an avenger to execute wrath on him who practices evil" (Romans 13:3-4).
The fact that your mother is paying fakes to pray for your brother tells me that she knows he is living wickedly. So the truth of the matter is that your mother is trying to bribe God to overlook your brother's sins and to not punish him as he deserves. But that very request goes against the nature of God. Prayers that are not according to the will of God are not answered. "Now this is the confidence that we have in Him, that if we ask anything according to His will, He hears us" (I John 5:14). Her heart is good in that she loves your brother, but she isn't thinking this through. I suspect that it is because your brother won't listen to her, so she is trying to find a way to protect him from himself -- something that cannot be done -- not without changing him.
I've showed you a number of verses and that is how I suggest you help your mother. When a point comes up, show her a verse. Let her see it for herself. You'll likely only get one verse in at a time because she'll shutdown when she sees the direction the Bible is going. But she also won't forget what she read and, hopefully, she'll begin to think about what she is doing.
"But avoid foolish and ignorant disputes, knowing that they generate strife. And a servant of the Lord must not quarrel but be gentle to all, able to teach, patient, in humility correcting those who are in opposition, if God perhaps will grant them repentance, so that they may know the truth, and that they may come to their senses and escape the snare of the devil, having been taken captive by him to do his will" (II Timothy 2:23-26).