The same thing could be said of racism though.
He never suggested eliminating religion would eliminated violence, but all else being equal, both racism and religion mean there is more violence.
Good people will always do good things, and bad people will do bad things, but it takes religion...