It’s important to remember: few outside of Germany knew that concentration camps existed, certainly not the scale of them or how appalling the conditions were. Consider the amount of information that Gandhi could reasonably have about activity in Germany and Europe. As far as he was concerned, the evil empire dominating his country was just having a costly spat with the evil empire dominating another country, sacrificing the welfare of his people for those of their neighbours in Europe.
Bullshit. People knew, they aren’t dumb. The polish underground reported on them regularly and even had operatives inside them. People living around them knew they were death camps. You cannot commit such slaughter and somehow magically hide it.
Average people simply didn’t have access to information at the scale we now enjoy at that time. Leaders of countries and militaries might know, but unless it was being reported by wire services and in local newspapers, the average person would have had no rational way of finding out about it.
People in Poland knew. That’s what I’m certain of. I know because my grandparents mentioned it and they were “average people”, far away from the camps.