If one builds a wall adjacent to a neighbor's windows, it must be built far enough that he not intrude on his neighbor's privacy (Bava Basra 22a, free translation).
In Jewish law, privacy is a right. As the above excerpt from the Talmud shows, a court can protect an individual's privacy.
Physical privacy is but one dimension of one's right; we also have the right to keep knowledge of our affairs away from the public eye. Not only can discussing or disclosing another person's affairs cause great damage, but in addition, it can intrude upon the other person's privacy. People have a right to their own thoughts and feelings, and this right to privacy must be respected, even among friends and family members.
Some people get offended when they discover that someone withheld personal information that they felt they had a right to know. Of course, while a person entering a partnership (whether business or personal) has a right to know certain things (such as the other party's past record of honesty), the other party certainly has the right to keep other things private.
Intimacy is a bridge between two separate people; only if we respect another person's right to a "self," a sense of privacy, can we expect intimacy to exist.