Those who give priority to their physical selves and make the soul subordinate cannot achieve sincere brotherhood (Tanya, chapter 32).
Rabbi Schneur Zalman states that a thorough unity is achieved between friends when their neshamos (souls) are permitted to fuse. Since all neshamos are part of God Himself, and inasmuch as God is the Absolute One, all souls can similarly be one. Separation and divisiveness among humans do not derive from the soul, but from the physical self.
The needs and desires of the physical self - the quest to satisfy one's earthly drives - are the causes of divisiveness. The neshamah does not seek pride nor wealth, is not offended, and does not seek to berate others. All these are traits of the physical self. To the degree that one recognizes the neshamah as one's true essence and subordinates the physical self thereto, to that degree one can eliminate the divisive factors and achieve true unity and brotherhood.
We thus see why spirituality is of such overwhelming importance. Hillel said that the essence of the Torah is "love your neighbor as you would yourself." To achieve such love, one must eliminate the impediments to sincere love of another, and as Rabbi Schneur Zalman stated, these impediments are the non-spiritual aspects of life. The greater the degree of spirituality one achieves, the more perfect can one's love of another person be.