The Saudi foreign ministry said the UN needs to be reformed first.
It said the Security Council had failed in its duties towards Syria as well as in other world conflicts.
Saudi Arabia has previously expressed frustration at what it sees as an international failure to act on Syria, where it staunchly backs the rebels.
Usually there is fierce competition to secure one of the 10 rotating seats on the UN Security Council – a chance to sit alongside the five permanent members for two years on the top UN body which rules on international security issues.
Instead Saudi Arabia has used the coveted position to lodge a very public protest.
This is the second protest from Saudi Arabia. Earlier this month the Saudi foreign minister pulled out of giving a speech at the UN General Assembly, expressing similar frustration.
It is a dramatic gesture, but whether it makes immediate reform of the Security Council more likely, as the Saudis demand, is probably questionable.
The announcement came hours after Saudi Arabia was elected for the first time to one of the 10 rotating seats on the Security Council.
The non-permanent members sit on the council for two years, along with the five permanent members – the US, the UK, France, China and Russia.
“Work mechanisms and double-standards on the Security Council prevent it from carrying out its duties and assuming its responsibilities in keeping world peace,” the Saudi foreign ministry said in a statement.
“Therefore Saudi Arabia… has no other option but to turn down Security Council membership until it is reformed and given the means to accomplish its duties and assume its responsibilities in preserving the world’s peace and security,” it added.
It said the UN had allowed the Syrian government “to kill its own people with chemical weapons… without confronting it or imposing any deterrent sanctions”.
The failure “to find a solution to the Palestinian cause for 65 years” had led to “numerous wars that have threatened world peace,” the foreign ministry added.
It also criticised the UN’s “failure” to rid the Middle East region of weapons of mass destruction, including nuclear weapons.