Neither. Logic, or nature as finny would put it. I guess you could put it into the category of society if you wanted to, but society does not determine them. Society can be just as oppressive as any government. People may be so oppressed that the rights are unknown to them. Rights are not given they simply exist. They can be protected such as in the US bill of rights, but that is not giving rights either.
Simply at the most basic form that people can do what they want without harming the rights of others.

__________________
May your paradise always be green, you liberties always be full, and may the ignorance of you enemies not drive you to be pro-nuke.

"We contend that for a nation to try to tax itself
into prosperity is like a man standing in a bucket and
trying to lift himself up by the handle."-Winston Churchill