Why did the dutch/british settlers in South Africa want or need to restrict rights of black africans?

They didn't restrict the blacks. When the settlers arrived most of SA was "no mans land". Because of them SA has an infrastructure. They saw vacant land and took it to farm on. Producing crops and jobs. They just wanted to protect what they had built.